ICC logo IFAS logo


ICC Meeting:

IFAS COMPUTER COORDINATORS
(ICC)

NOTES FROM December 12th 2008 REGULAR MEETING


A meeting of the ICC was held on Friday, December 12th, 2008 in the ICS conference room. The meeting was chaired and called to order by Steve Lasley at about 10:00 am.

PRESENT: Fifteen members participated.
 
Remote participants: Micah Bolen, Dan Cromer, David Bauldree, Chris Fooshee, Louise Ryan, Mitch Thompson and Wendy Williams
 
On-site participants: Bill Black, Andrew Carey, Francis Ferguson, Diana Hagan, Winnie Lante, Steve Lasley, Kamin Miller and Mark Ross.
 

STREAMING AUDIO: available here. Steve once again forgot to start the recording immediately. In fact, the first 20 minutes or so were lost at the beginning -- sorry once again.


NOTES:

Agendas were distributed and the sign-up sheet was passed around.


Report from the chairman


Member news:

New members...

M Stewart Collins isn't new, but Steve recently learned that Fisheries and Aquatic Sciences negotiated that he provides their IT support as part of their merger with SFRC.

Departing members...

Steve recently learned that Mike Stewart is no longer with the Program for Resource Efficient Communities, John Dixon is no longer with WFREC and Stephen Reese is no longer with FRE.

Recap since last meeting:

As per his usual procedure, Steve pointed folks to the notes of the last meeting, without going into any details.


Policy


Shibboleth and Identity and Access Management (IAM) at UF (see prior discussion)

Steve had heard nothing on this and no one had anything else to relate at this time.

Report from the December ITAC-NI meeting

This meeting was cancelled. The next meeting will be in January. The regular meeting time has been moved back 30 minutes to 1:30pm at Dan Cromer's suggestion, so these will be on the second Thursday of the month at 1:30-2:30pm. Note that they are always available via Polycom and streamed from the bridge.

UF IT Action Plan

Steve took a quick run-through of the UF IT Action Plan site. Kamin Miller mentioned having attended Wednesday's Open Forum for Faculty and Staff and a number of us attended yesterday's similar session for IT Staff. Steve mentioned concern there over whether our jobs would be secure should restructuring occur. Dr. Frazier said that our financial situation was more of a concern there. He stated that he believed IT was understaffed and underfunded in general, however, and that he did not see this action plan as contributing to any layoffs.

We discussed when we should invite Dr. Frazier to the ICC and decided that Steve would try to get him there for January. Steve also proposed we plan to skip the February ICC meeting due to the planned 2-day seminar on Windows Group Policy by Jeremy Moskowitz on February 12-13th.

Comprehensive IT risk assessments will be REQUIRED soon

The deadline for this has been moved to year-end.

Update on changing the Barracuda default settings
(see prior discussion)

Steve has been doing his best to get this resolved with the help of the ICC. A handful of IT support persons have still not responded, however, most of the clients have been or are being contacted:

Project Status

At Wednesday's UF Exchange meeting, Dan Cromer indicated to Mike Conlon that we are ready for the defaults to be changed. Dr. Conlon responded that this would have to wait until after the holidays because he wanted to make sure that all tier-1 and tier-2 staff were available to assist with any problems.

UF Exchange Project updates (see prior discussion)

Resource scheduling in Exchange Server 2007

Dan Cromer mentioned that the UF Exchange project group is working on implementation procedural details for resource scheduling. Steve didn't have Outlook 2007 installed so was unable to assist Dan in demonstrating how resources can be added to meetings; but the resources themselves are/will be visible in the GAL (named with an "@" prefix) and rooms can be added to meetings in Outlook:

Scheduling Rooms

Student mail to move to Google Gmail?

Kamin Miller related after the meeting that there was an announcement at Wednesday's Open Forum on UF IT for Faculty and Staff that UF has made the decision not to move students to Google. There is some indication that they will permit students to forward to their own personal accounts. This likely makes sense since they can just "pop" their Gatorlink accounts from Google in any case.

WAN transition to CNS (previous discussion)

Bill Black and Francis Ferguson both expressed concerns over the status of the UPS's on our switches at numerous CEOs. IFAS IT is negotiating with CNS for their replacement but there seems to be some disagreement on who should pay for those. IFAS thinks CNS should cover this out of what we pay them for support but CNS is short on funds--particularly in this first year.

Split DNS solution for UFAD problems

Steve wants to keep this on the agenda for future reference.


Projects


IFAS WebDAV implementation

There continues to be no progress on the documentation which was to happen prior to announcement. Since this has never been formally announced, the matter remains on the agenda as a standing item.

Vista Deployment via SMS and WDS

Nothing further was available on this topic at this time.

Exit processes, NMB and permission removal (prior discussion)

Nothing further was available on this topic at this time.

Re-enabling the Windows firewall

This is still planned but is pending the time to implement.


Operations


DHCP log access

Steve had indicated via the ICC-L back in early November that ICC OU Admins can now investigate UFIRT P2P notices (track down hostnames and hardware addresses) without having to bother Wayne Hyde. Steve had documented access and usage on the secure portion of the ICC web site (if-admn credentials required for access).

Chris Leopold has now made this even easier by writing an application which may be used to access these logs.

Disabling/deleting computer accounts based on computer password age

As with so many things in these times of inadequate staffing, finding time for implementation is proving difficult.

New MPS/DC testing -- access by unit-level administrators

Andrew is still working on Hyper-V but has been very busy doing general support and has been unable to make the kind of progress he would like.

Andrew mentioned that our DCs are beginning to fail at an increasing rate so we may need to speed up our deployment plans a bit.

Report generating system

Unfortunately, this is yet another useful project for which implementation time has been lacking.

Core Services status

Andrew reported that all units are now migrated to the new file cluster. Steve asked Andrew if he could forward the smb: path which our Mac users may use for connection and Andrew indicated that he would.

ePO version 4 status

Wayne was unavailable to provide any updates on this topic. He has been out sick recently.

Status of SharePoint services (prior discussion)

Steve wanted to document the problems which occurred with SharePoint November 21st. Ben Beach had supplied the following thoughtful and responsible explanation to some of those who were adversely affected by this unfortunate occurrence:

I am sorry for any confusion, loss of data, wasted man-hours, 
and any inconveniences that our outage caused. I want to be 
completely honest with everyone and leave no doubt. This is 
what we know in regards to our downtime on Monday:

Of the ten operational SharePoint sites we currently have in 
use, four were wiped out on Monday: My IFAS home page, Admin, 
Centers, & Research. What caused them to be deleted we are 
not sure of. The databases for each of these sites was still 
present, but the data that was contained in those databases 
was gone. The backup procedures we had in place failed. 
This is what we think happened:

   1. The SharePoint environment was setup to have a feature 
known as Site Use and Confirmation. This is an automatic 
process that monitors each site and if it finds the site has 
had no changes in 365 days, sends email notification to the 
owners (28 times in consecutive days, one a day) of the site 
to verify whether the site should be saved or deleted. If 
the Owner wants to keep the site, they click a URL link in 
the email and that confirms to keep the site. If the Owner 
does not reply, the site gets automatically deleted after 
the 28th notification. I am listed as the Owner, however, I 
never received an email requiring me to confirm the site 
use. We checked all deleted files, Junk folders, SPAM & 
Barracuda folders, and the blocked email folders but could 
not find any email notifications. The reason we suspected 
this was I had received 4 such emails on 10/24/2008 for four 
of the sites that did not go down and confirmed them. We 
wanted to make sure there were no missed emails that I had 
missed. We could not find any. If this had happened, then 
the site & data would have been deleted just as our 
environment had. That was why we questioned this first. This 
feature was enabled to automatically clean-up our SharePoint 
environment so we did not have a lot of unused subsites or 
webpages build up in our environment.  

   2.The backups on our SQL server were in place on our 
original SQL server and the SQL DB had recently updated the 
SQL environment to a new server. When he did this, the 
backup processes were supposed to migrate with the databases. 
Apparently they did not. So the most recent backup we had was 
from my backups when I migrated our environment from Windows 
Server 2003 to Windows Server 2008 last month. So we lost 
changes all the way back to 10/20/2008.

   3. As such, with this last calamity, these are the 
measures we have taken to ensure this does not happen again:

      a. The Site Use and Confirmation feature has been 
   turned OFF. It will still monitor the process, but the 
   sites will not be automatically deleted. The Owners will 
   have to do that.

      b. The SharePoint backup has been re-configured to 
   backup our environment every Wednesday night at 12:00 AM. 
   I am going through and manually backing up each site 
   every Wednesday & Friday night until I am positive we 
   have a valid backup schedule and it runs automatically. 
   This gives us a two  three day backup schedule. After we 
   have determined we can depend on these backup procedures, 
   the backups will take place once a week. 

      c. The SQL DB has re-instated the backup procedures on 
   his SQL server and monitoring every time it runs to 
   ensure it has run successfully.

      d. We have gotten our AD administrator involved and he 
   is testing a SharePoint backup procedure through our 
   Veritas backup server. This feature he is testing will 
   cost our dept. roughly $500 but if it works, will be well 
   worth the expense.

That was the bad news. The only positive thing I can share 
with everyone is that our design actually worked as it was 
planned on for using our environment.
Each site has a separate database and was intended to be 
used if another site went down. Well, our home page, 
http://my.ifas.ufl.edu was down but the subsites were still 
functional (such as http://my.ifas.ufl.edu/sites/depts/AEC). 
In a normal webpage, that would not be so. And no other 
site that was still operational had any lost data.

I know that this causes doubts in the security of our 
environment, and will cause users such as yourselves to 
question the security and dependability of this SharePoint 
implementation, as well as my administration capabilities.  
All I can say in defense of it is: this is the first time 
we have had anything go down in our environment since it 
was started two years ago. We actually setup 
http://my.ifas.ufl.edu in January of last year (2007) and 
had been testing it and designing it until we started 
rolling it out in January of this year (2008), and never 
had any data or site lost in that entire time. We had it go 
down twice but those were DNS server issues, not SharePoint. 
Once the DNS server was brought back on line, SharePoint 
sites were working.

I am confident we will not have this problem happen again. 
Again, I am sorry for any trouble and inconvenience.

Andrew added that they are investigating the use of Backup Exec for SharePoint and that option looks to be a good one.

Public folder file deletion policies and procedures status

Steve raised issues he saw with using SharePoint to replace the public folder. He is concerned that we would just be moving and propagating the problem. Since there currently isn't any SharePoint location for file sharing across all our various branches, individual units would have to create those. Then we would have a growing number of places where we would have to be concerned regarding inappropriate sharing of protected data.

Andrew responded that the planned web interface is likely a better solution, where a file would be uploaded and available temporarily via an obfuscated URL. The problem there is once again finding the staff time for implementation.

Videoconferencing topics

Dan Cromer mentioned that Patrick Pettus continues to work on the Tandberg Management software. It has the potential to save a lot of leg work in monitoring things and keeping them updated.

Patching updates...

Microsoft

There were six critical and two important Microsoft patches for December.

Third-party apps

Version 6 update 11 of Java was released and it does indeed correctly patch update 10 in-place. Update 11 addresses a number of recently reported vulnerabilities. Third-party vulnerabilities remain the biggest and growing concern. Secunia's vulnerability scanning is highly recommended to locate and patch such issues. Their online scan may be used here at work, but Secunia PSI 1.0 should be recommended to folks for use on home machines.

MS Office News update

There were no new updates to give at this time.

Job Matrix Update status

Steve wants to leave this matter as a standing agenda item for future discussion.

Remedy system status

Steve wants to leave this matter as a standing agenda item for future discussion.


The meeting was adjourned rather early at about 10:55 AM.