ICC logo IFAS logo

ICC Meeting:



A meeting of the ICC was held on Friday, November 12th, 2010 in the ICS conference room. The meeting was chaired and called to order by Steve Lasley at about 10:00 am.

PRESENT: Fifteen members participated.
Remote participants: Bill Black, Micah Bolen, Dan Cromer, Francis Ferguson, Kevin Hill, Chris Leopold, Marvin Newman, Nick Smith, A. D. Walker, John Wells, and Wendy Williams.
On-site participants: Dennis Brown, Andrew Carey, Wayne Hyde, and Steve Lasley.

STREAMING AUDIO: available here


Agendas were distributed and the sign-up sheet was passed around.

Report from the chairman

Member news:

The only membership news that Steve had to relate was that Tom Barnash is apparently working part-time for a faculty member in Fisheries now in addition to his regular position with WEC.

Recap since last meeting:

As per his usual procedure, Steve pointed folks to the notes of the last meeting, without going into any details

Special guest: Iain Moffat

UF's Microsoft Applications Group dissolved? -- now under OSG

The Microsoft Applications Group (MAG) was informed of organizational changes on October 27th. Initial reports indicated that MAG had been dissolved and that its staff would report to OSG. This change was of great interest/concern because IFAS depends heavily on the various centralized Microsoft services; questions immediately arose as to what this change might mean for those services both in the short- and long-terms.

Public announcement was provided the following day...

Message to the ActiveDir-L list from Barb Sedesse:
"MAG transition into CNS Open Systems" Thu 10/28/2010 3:08 PM

On July 1, 2010, the Microsoft Applications Group (MAG) was transferred by the CIO to CNS. The MAG has been reporting to me for the last 4 months. I was asked to do 2 things -- • keep everything up and running in the interim, while assessing MAG services • recommend the best organizational fit for MAG services over the long run

The evaluation has now been completed and approved by the CIO and his leadership team. As a result, effective immediately, the members of the Microsoft Applications Group will become a part of the CNS Open Systems Group, which is managed by Iain Moffat.

For those users, system administrators, and other constituents who appreciate and rely heavily on the efforts and products supported by the former MAG, we want to assure you that the support and communication channels that you are used to will continue. The Tier 2 meetings, coordinated by Academic Technology, play an extremely important part in all of this, and will continue to be a valued source of information sharing. In addition, the technical leads for each of the services will continue to lead those areas. Buck Buccholz will remain the lead for Sharepoint, Luis Molina for Exchange/OCS and Mike Kanofsky for Active Directory. Erik Schmidt will be responsible for related planning, promotion, and partnership initiatives, such as helping to maximize the value of our Microsoft Campus Agreement.

As a part of the overall realignment of central IT, this change was made in order to consolidate like functions and thereby improve efficiency, reliability, and redundancy in central services. Over time, we will continuously improve the former MAG services through the sharing of resources.

If you have any questions or concerns regarding the continuity of services during this transition, please contact Iain Moffat or me directly.

The ICC has had a long-standing hope (since at least the end of 2004) that UF's centralized Microsoft platform support could be organized as a single, separate unit, similar to OSG at CNS. Perceived incompatibilities between Open Systems and Microsoft philosophies made the announced restructuring surprising to many, especially given the disparate hardware and software platforms involved. It was difficult to guess what improvements this move might make to "efficiency, reliability, and redundancy".

Additionally, the timing of this change was puzzling in that it happened just prior to the institution of our new IT governance structure. It seems like the importance of these services would have warranted more broad public discussion prior.

Iain reported this morning he would be unable to attend

In order to help answer our questions and concerns, Steve had invited Iain Moffat, Manager of OSG, to visit with us. Unfortunately, Iain let Steve know earlier in the morning that he would be unable to attend due to reasons outside his control. He sent his sincere apologies and said that he would put our December meeting on his calendar.

Will this signal a change away from primarily cross-platform solutions from OSG?

In Iain's absence various ICC members related their feelings and concerns about this. Steve mentioned the example of OSG's current file sharing offering; this supports CIFS on Linux but may not offer all the Microsoft file sharing services such as we have become accustomed to--with features like DFS, Volume Shadow Copy, and detailed usage reports to mention just a few. IFAS also intends to move to Microsoft's file server platform in our next iteration of the file cluster--something that would provide single instance storage. Will future services be limited to least common denominator on the premise of cross-platform compatibility or will we be able to continue to build the services specific to our platform of choice?

Concerns over OSG's reputation regarding MS solutions

Kevin Hill said that when he thinks of OSG he thinks "Microsoft haters". Back when IFAS was trying to organize a centralized UF Active Directory Kevin noted that there was a lot of resistance from OSG. Kevin is not comfortable with that same group holding the reins on directions for Microsoft platform development on campus. Kevin said that he is beginning to wonder if he did the right thing in joining the UFAD domain if we don't have a group that is driven and supported for Microsoft platforms. It is a great concern of his.

OSG obviously has not had much expertise in Microsoft products prior to this reorg. Steve thought it would be good news to hear that they were going to use this move to incorporate and grow that--if such is the case.

SLA concerns

Kevin mentioned that we had a SLA with the UFAD group which later became MAG. Since MAG no longer exists, Kevin wanted to know who that agreement lay with now and whether or not it needed to be re-written.

Feedback offered so far

Steve noted that there had been at least one tactful message from the Law School sent to the CIO seeking more information about this sudden change and that Steve had encouraged Dan Cromer to respond similarly on behalf of IFAS. Dan had said he responded privately to Tim Fitzpatrick, Barb Sedesse and Iain rather than to do so publicly.

Recent Exchange meetings have cast some positive light

Andrew Carey commented from what he had heard at recent Exchange meetings [Note: those are now being made available via LiveMeeting. A Tier 2 portal is available and Kiem Tran should be able to provide more info]; Andrew has been attending those recently because the UFAD meetings have become few and far between and Andrew learned that UFAD news was being incorporated into the Exchange meetings now.

Iain and James Oulman were at the meeting two weeks ago; Andrew believes it was the first time Iain had ever attended. They both seemed to be interested in continuing the same support structures which had been in place prior.

Barb Sedesse was in the Exchange meeting observing this Wednesday as well. Part of the discussion pertained to moving away from the Barracudas to just ProofPoint--this is something James raised. Andrew was pleased to hear Fran McDonell of the UF Computing Help Desk speak up with what they needed for this change to happen--as opposed to CNS dictating what was going to happen. Fran was making sure that we had the documentation and support in place prior to any major changes. This increased back and forth between CNS and the rest of the IT community hopefully bodes well for all.

Andrew also mentioned that Tim Fitzpatrick and Barb Sedesse have plans to discuss this matter at an upcoming UF Exchange Advisory Group meeting in December. They are apparently doing a lot of work to assure folks that the Microsoft services will not be going away.

Answers pending on how the consolidation might be expected to improve things overall

Steve said that he would like to hear more about the plans to "consolidate like functions and thereby improve efficiency, reliability, and redundancy in central services." Other than removing the Barracuda, Steve isn't clear on how such efficiencies might be obtained. He wants to know if the OSG Gatorlink email group will be cross-trained on Exchange or if the former members of MAG will continue to collaborate among themselves.

Concerns over a loss of synergy among those supporting Microsoft platforms

Kevin commented that he isn't so much concerned that existing services would be terminated; they have too much momentum for that to occur. [Andrew believes the former MAG group has upwards of 75% adoption of one or more of their services across the various groups on campus.] But when you are dealing with a platform as large as Microsoft with the variety of offerings they have, there is a certain synergy that has existed between the various support staff for centralized Microsoft services. Having all those folks working together in the same room on the same sorts of things has yielded amazing things in the past. If you look at that from a service standpoint and throw Luis Molina in with the Gatorlink email group you threaten to break up that synergy. Kevin feels that interaction which be sorely missed if this restructuring indeed changes things.

In an email response to Steve, Iain had mentioned that he knew "how important all of the services are to all of you and our only immediate goal is to provide better coverage and tighter integration with other campus services." The question is how will that integration occur?

Will this affect consideration of other Microsoft-based solutions at UF?

Another question is how open the OSG would be toward future expansion of Microsoft technologies. One example that comes to mind is Direct Access. That service has great potential for our Windows clients, but will take considerable work on the server side to implement. How willing will OSG be to promote such technologies, where would such decisions be made, and how can Microsoft users in the UF community get some input into those kinds of things? Steve hopes to eventually hear from Iain about the past philosophy of his group regarding Microsoft and whether or not this move will mark a change. If OSG can support and continue to nurture the growth of Microsoft platforms then perhaps this move will be eventually be seen as a good thing overall.

Cost disparity within OSG hosting options

Kevin noted that OSG does offer web and database hosting on both Linux and Microsoft platforms with the latter being considerably more expensive for some reason. Steve noted that Wayne Hyde has been trying to keep a handle on what IFAS services would cost under CNS's current charging structure. Wayne said that the CNS model is pitched more at single faculty members who want a web site or a database. For individuals, their pricing probably makes good sense. Since IFAS already has a DBA and the hardware to run it on, it would cost IFAS a lot more to outsource things to CNS. Both IFAS and CNS have virtual infrastructures, but CNS also has duplicate SANs that are mirrored off-site and replicated to Atlanta. Main campus will be replicated to the East-side campus before long as well. Those are high-end services which IFAS cannot afford to deploy in-house. Until CNS's cost model changes, however, it doesn't make sense for IFAS to move many of its services centrally. It would cost about $100,000 to have CNS host our SQL databases, for example--not counting the cost of backup. We just bought a new SQL cluster for a total capital cost of less than $40,000.

Might cost models be changing?

Steve related that the new CIO had mentioned at the last ITPAC meeting that he wanted to get away from charge-back--preferring to have things paid off the top. Maybe changes will be coming that make centralizing more things attractive monetarily; we need to continue to be cautious to other factors such as support, flexibility, and manageability, however.

Potential option for input from the ICC

Dan Cromer mentioned that a message from ICC chair to Ian, with copy to Tim Fitzpatrick and Elias, would be an opportunity to express concerns. Steve said that he would try to draw up something like that from these notes and run that past the ICC before proceeding.

Videoconferencing and WAN discussion

[In order to make meeting participation more efficient for Patrick Pettus and James Moore, these two topics have been moved to reside at the top of our agendas.]

Videoconferencing topics (previous discussion)

Hardware changes to VC endpoints

Video Services recently put out the word that changes affecting videoconferencing endpoints (hardware) should be reported in advance (before the changes are made) to vc-hw-change@video.ufl.edu.

Such changes include, but are not limited to:

  • Network changes (IP address, ISP, NAT, firewall, bandwidth, etc.)
  • Equipment location (including removal)
  • Equipment replacement (including new endpoints)
  • Connected equipment (cameras, monitors or any other device requiring configuration changes)

Dennis Brown noted that he had reported a change via the above address and Lance Cozart had responded immediately.

Polycom maintenance renewals

No updates available...

End-user Scheduling

No update available...

New end-point options?

In talking with Lance Cozart, Steve learned that he has been investigating Tandberg endpoints at Dan Cromer's request. The main point that stuck out with Steve about that discussion was the cost; unless some steep discounts can be arranged, Tandberg units seem prohibitively expensive.

Lance has a Tandberg C90 and a C20 there in building 162 for the next week or so if anyone wants any training or to compare the difference between Polycom and Tandberg. The Tandberg units especially the C90 and C60 offer video compositing which is great; you can take 13 inputs and your callers and move them up to 4 monitors but these units are more expensive due to the extra switching. Also these units can do mix minus and have virtual video and audio programming for connecting inputs to outputs.

The C20 is a good starting point. It lacks the ability to separate content audio and near far audio. The C20 has less features that the HDX7000 Polycom. The C20 is not an integrator series. Some cables have to be custom made. The serial cable for the C20 only comes in shorter lengths. Tandberg does not sell longer serial cables for the camera. The mic cable is also a special cable. The C40 is a better choice for installation but price is around $13,000. Lance is waiting on CCS to send us pricing, but the C90 retails for a steep $50,000.

Dan Cromer mentioned that Allan Burrage at CREC is looking into a Polycom QDX 6000 system (an HD 720p system) which Dell is apparently selling for $2849. If that model turns out to be useful it would make a new price point for endpoints.

New Polycom directory

As you may be aware Tom Hintz used to provide updates to the directory files for PVX and VXS systems prior to his leaving. Lance Cozart is now looking into updating those but could use everyone's help. Please check out the Excel spreadsheet which he has posted on SharePoint and let him know if the machines under your purview are listed correctly. Lance would also appreciate volunteers to help him test these connections--so please consider getting with him to lend your help.

Office Communicator infrastructure status (previous discussion)

Dan Cromer related that with its next version OCS is being renamed to Lync.

Dan had shared a link to a Polycom webcast on Delivering on the Promise of Unified Communications which mentions their recent agreements with Microsoft.

New VC gateway status (previous discussion)

Update as available...

Recording lectures for Distance Education (previous discussion)

As was discussed on the Accordent-L list, new skins are available but one apparently has to submit a ticket to Accordent support to get those.

Steve has created a web page that points to the various accordents across IFAS. You can use that to see how much (and what kind) of use those systems are providing. It would appear to Steve that we currently aren't getting good use out of this investment overall and should likely look at why. Many sites still look to be improperly configured; Steve noted that at least one site replaces the video with content when content is being sent. Others clearly have not been used at all and one site needs to flip their template so that the speaker appears to be looking at the content and not away from it. Dennis noted that his system has had a number of issues, including one where the recording would just stop soon after it was begun. Marvin Newman reported that his unit had to be returned due to a bad HDD and it was not yet in production.

WAN transition to CNS (previous discussion)

Connection of UF and IFAS Remedy systems with the CNS Remedy system

No update available...

Updates from James Moore

James did not attend this month so there was nothing to report from the WAN group. Wayne Hyde wanted to mention, however, that some remote servers are currently hooked to switches via 10/100 ports; he would like to investigate and ensure that free 10/100/1000 ports aren't available in such cases and aren't just being overlooked.


IFAS IT is back on the IFAS Org Chart!

Since the merging of IFAS Communication Services, External and Media Relations, and Information Technology August 2009, "IT" has been pretty much out of the picture with regards to the IFAS Organizational Chart. While that didn't greatly change how things worked in actuality, Steve is very pleased to note that IT has now been pulled out of that again and Dan Cromer now reports (again) directly to Joe Joyce. Steve assumes a new Org Chart indicating this arrangement is in the works.

Alternate IFAS domains in e-mail

Steve wants to keep this on our agenda for future discussion. He believes there is no advantage to having multiple aliases and that we should move towards removing those if possible.

Identity Management (IdM) Interface Training

Steve wants to remind everyone of the "UF_PA_IDM_NETMGR" role which will allow you to set NMB for your users. Your Department Security Administrator can do that for you.

Sakai e-Learning System now in production (previous discussion)

Steve noted that one of his faculty did run into a quota limit when uploading files for her course. The unhelpful error message was "an error occurred copying some or all of the selected files". She determined the reason only by call the UF Computing Help Desk and asking to be referred to Academic Technologies Sakai support on the issue. Her quota was then raised though there was not mention of what the defaults are or what levels would be acceptable; those details are likely still being worked out.

In an additional turn of events, there seems to be some evidence that access policies are changing. Chris Fooshee reported that it appears Sakai is no longer available to Non-Gatorlink holders having received the following:

Hello Chris - You were able to add "non-official participants" but they weren't actually able to login because they had no password. We removed that option because it was creating problems with people thinking they had given access to someone when they really hadn't. Only people with gatorlink usernames can access Sakai. Unless your department can grant temporary gatorlinks to these people, you won't be able to use Sakai. Let me know if I can be of further assistance.

JoLaine Jones-Pokorney
eLearning Support Services

This was very disappointing to Chris as "several of our faculty members are currently working on cooperative research projects and multi-disciplinary committees between colleagues at other universities and private companies. I can see significant value in the Sakai tools for collaborative work among various outside groups or committees at multiple universities and private corporations. Since it would be impractical to establish GatorLink accounts for such users."

Steve and Chris had discussed the matter with Dan Cromer via email and Dan mentioned he would ask Fedro Zazueta if he would turn the non-GatorLink access back on until LOA-1 is available. Apparently, Dan has yet to hear back from Fedro on the matter.

Steve had speculated how we (end users and IT support) might get input in such changes. The new UF committee structure offers several future possibilities. Dan then pointed out that he would hope most such things could be negotiated directly with the service providers and only brought to the committees if that did not prove fruitful. Steve then pointed out that he didn't believe most end users (or even IT support staff) are aware of how to make such cases/requests to the various service providers—or are even necessarily aware what services are provided by which groups. Perhaps creating a uniform route (website?) for such input/feedback across the various centralized services would be worth consideration by some committee or other? Of course, it would take staff time to respond as well, but that might go a long way towards plugging the client base into day-to-day IT operations. That would be valuable for both the users and the providers as well I’d think.

myuf Market (previous discussion)

Steve wants to keep this on our agendas in case discussion seems warranted.

UF Exchange Project updates (previous discussion)

Note that there was some discussion of UF Exchange this meeting in the early section on the changes in the UF MAG group.

UF Exchange upgrade

Status report not available...

Barracuda load issues

Update not available...

Centralized FAX service via Exchange (previous discussion)

Update not available...

Split DNS solution for UFAD problems

Steve wants to keep this on the agenda for future reference.

IT survey is coming (previous discussion)

Dan Cromer reported that this survey is apparently still being developed. Rob Adams, the new Chief Security Officer is now getting involved and it may be accomplished more by network survey than by manual survey, with additional input later.

Outsourcing of DE course development (previous discussion)

Update not available...

Negotiations underway for the Microsoft Campus Agreement

Dan Cromer related that he had not heard yet how this is going. Of course, our two big concerns are the continuation of paying server licenses off the top as well as the eCALS which permit us to use OCS, SharePoint and many other services--potentially Forefront Edge Protection for example. The last time Dan spoke with Elias Eldayrie about this he was apparently working on getting Shands and UF students included as well.


IFAS WebDAV implementation

There continues to be no progress on the documentation which was to happen prior to announcement. Since this has never been formally announced, the matter remains on the agenda as a standing item.

Windows 7 Deployment via the WAIK, MS Deployment Toolkit 2010, USMT 4.0, WDS, and SCCM

Windows 7 deployment

update not available...

Exit processes, NMB and permission removal (prior discussion)

Nothing further was available on this topic at this time.

Re-enabling the Windows firewall (prior discussion)

Update as available...

Services Documentation: Is a Wiki the way? (prior discussion)

*** Food!    Prizes! ***

A Wiki Wag Geek Gathering to promote the UF IT Wiki is planned at 2:30 PM in Turlington Room 2215. Micah relayed that remote participation is welcomed as well.


Restoration of back-ups on the file server

Wayne Hyde intends to document and announce proper usage as time permits.

Membership of ". IFAS-ICC" email distribution group to be narrowed to ICC members only (previous discussion)

Steve will keep this as a standing item on our agendas for now as a reminder. The ICC distribution list is more targeted and restricted to IFAS IT support folks only.

IFAS efforts toward Green IT (previous discussion)

Update as available...

Creating guest GatorLink accounts: singly or in bulk (prior discussion)

Steve had left this on the agenda in case further discussion was deemed warranted.

Can IFAS support DirectAccess in the future? (prior discussion)

Steve wants to keep this topic on our radar.

Moving away from the IFAS VPN service (previous discussion)

Steve assumes that moving our VPN to private IP is waiting on Wayne Hyde finding the time to implement.

VDI desktops as admin workstations (previous discussion)

This is another cool service that Wayne has in progress and which is awaiting sufficient time to pursue further.

Wayne's Power Tools (prior discussion)

There was nothing new to report this month.

OU Technical Contact email groups now in use

Steve had mistakenly assumed that automatic FSR reports had been configured for all OU Admins earlier, but it turns out that he was just an early "beta-tester". In any case, Wayne is still working to get those reports going out to all units--as he described via e-mail:

Message to the ICC distribution list from Wayne Hyde:
"FSRM reports for campus file server" Wed 10/13/2010 11:26 PM

Some of you may notice emails from FSRM@ifas.ufl.edu in your inbox starting today. Do not be alarmed. Do double-click on them and look them over. Much of the information is just that – informational – and meant to give you a better overall view of your unit’s data usage on the campus file server. There will be quite a bit of valid data included in these reports, so do not just delete the 20GB file because it is listed in the large files section. However, do try to eliminate wasted space where possible.

There are 4 reports for each unit:

    Large Files, Files by Group, Quota Usage, Duplicate Files

    Files by Owner

    Files by Owner

    Files by Owner

The file groups in the “Files by Group” section include ISO, PST, BKF, and movie files (avi, xvid, etc).

Large files are files over 300MB in size.

Since FSRM has some limitations on the number of groups and matches it will include in the report I had to break it up a bit. You’ll understand once you look at the reports. The subjects on the emails will not be entirely descriptive either, and you can thank MS for that. I plan to have these reports processed and emailed once a week. They take quite a bit of time to generate on the server side and hit the disks fairly hard during the processing, so I will most likely generate the reports on Sunday afternoons.

The low hanging fruits are PST, BKF, and very large files that should not be on the file cluster. Examples of these files include backups of hard disks (Windows Backup dumping data to the network shares) and copies of every installation ISO possible.

The duplicate file report section is handy to find those users who create multiple folders that store backups of the same files.

The reports list the physical path to the data on the file servers and not the DFS namespace paths, so you most likely will not be able to click on any of the links as the UNC pathnames embedded are not correct. A typical error will be:

Broken links in FSRM reports

The error window above references IF-SRVCN-FILE1 which only the ITSA group can access. However, you should be able to easily identify the path to the files and get to the locations using the DFS namespace such as \\ad.ufl.edu\ifas\AGR or the direct path such as \\IF-SRVC-FILE1\DATA-L\AGR.

Wayne still has a bit more work to do configuring the timing of those reports; considering how hard they will hit the file cluster they have to be carefully staggered across the various units.

Policing of the various Unit folders should be a priority of each unit's OU Admin

Wayne is also considering creating separate reports for the unit folder. In the case of many units, users are still using those areas inappropriately and Wayne feels the need for better management. He is considering making those read-only to most and allowing just a few users write access. Steve pointed out that such a scenario would not work in the case of his department, however. Steve has provided a "Dropbox" on his own server for many years and plans to move that to the Unit folder when the server finally gives up the ghost. For that to work, Steve needs everyone in his OU to have write access. He has, however, initiated plans to make sure that this area stays clean; he plans to wipe it clean weekly and has created a file structure within it whose names clearly spell out that policy.

Wayne said that creating a similar file structure for each Unit folder which described proper usage might be his first step in getting things under control. User education by each OU Admin is an important and necessary component of all this. Steve would like to note that having more units using Folder Redirection of the user Documents folders might help things as well. Doing that naturally encourages correct file storage placement in its most secure location.

Computer compliance tool in production (previous discussion)

Update as available...

Folder permissioning on the IFAS file server

You are reminded to please take the time to read and implement the new standards. If you have any questions get with Wayne or Steve.

Disabling/deleting computer accounts based on computer password age

This is yet another matter for which finding time for implementation is proving difficult. Steve wants folks to remember that Andrew Carey has a good plan for dealing with this which he simply has had no time to address. In the meantime, it would be very good of each OU Admin to consider mimicking the proposed plan manually by keeping their own records and deleting any computer object which have been disabled for 90 or more days; Wayne's Power Tools can identify those. Steve has finally begun doing that for his own unit and it has made his view within ADUC much more agreeable.

New MPS/DC deployment

Andrew reported that Fergie's District 3 is fully deployed (new hardware with DCs up and running) and Ben Beach's District 2 (one of our larger districts) has about four servers remaining to be deployed. The MPS servers are to follow, but they are focusing first on getting the hardware deployed in most cases. This method was chosen because the old DCs are dying at a rapid rate and Andrew is running out of spare parts; the old MPS servers on the other hand have proved to be fairly reliable and he still has a pretty good supply of spare parts for those.

If there are sites that are in particular need (MPS full for example), please get with Andrew so he can adjust the order of rollout to best suit overall needs. One such example is at Indian River REC where they plan to consolidate a number of existing servers into the new MPS. Chris Leopold will soon begin working with Marvin to get that going.

They have begun now on Bill Black's District 4. He has four servers in his possession and will be picking up the last six the week of Thanksgiving. Kevin Hill's District 5 is up after that, so Andrew asked him to begin thinking about how that might best proceed. John Well's District 1 will be the last District to be deployed.

Francis Ferguson (Fergie) took the time to thank Chris Leopold and Andrew Carey for making the deployment of the new DC hardware go so smoothly.

Wayne mentioned that DPM includes an agent that might allow ITSA to backup certain VIP desktops as well in the future.

Core Services status (previous discussion)

Data Protection Manager status

Due to some unexpected issues, Wayne has moved ahead with performing Volume Shadow Copies for FILE1 from the new DPM server:

Message to the ICC Distribution list from Wayne Hyde:
"Campus File Server VSS change" Mon 10/25/2010 4:41 PM

Volume shadow copies on FILE1 were deleted due to a bug in Server 2008 where the file cache may eat all of the RAM in a server (http://support.microsoft.com/kb/976618). All is not lost, however, as we have been using DPM2010 to protect the cluster since October 3.

I have disabled shadow copies on FILE1 which will allow clients to restore previous versions out of DPM ahead of schedule. Nothing needs to be done on the client side; when a user attempts to restore a previous version (ie: right-click -> restore-previous-version) they will automagically see the previous versions of files saved on IF-SRV-DPM-C1.

The OUs that have been switched over include:

  • FSHN
  • ANS
  • ICS
  • AGR
  • CALS
  • SWS
  • ENH

The DPM server shadow copy schedule is set to take snapshots every day at 7:30am and 12:30pm.

It was always the plan to move shadow copies over to DPM to reclaim SAN storage space for our virtual environment. “Stupid Windows Tricks” aka “and….. your shadow copies are gone!” simply forced my hand and advanced the timetable. FILE2 has not been switched over yet, but will be soon enough.

There is another slightly annoying aspect of this change in that DPM apparently doesn't handle access based enumeration (ABE) for previous version shares, so one can see subfolders that you may not have access to within the previous versions dialog.

Wayne reported that the DPM server is now available in a read-only fashion via UNC. This provides one means of disaster mitigation; should the file cluster die the file server data would still be available during what would necessarily be a lengthy recovery process.

DPM now in use backing up remote sites

Wayne reported that so far there are four remote sites that are being backed up to campus: Plant City, Apopka, Fisheries, and WFREC. The plan is to begin this with the fast sites and gradually work through to the more poorly connected ones.

For those slower sites they have purchased five 1TB external drives which will be distributed to the District Techs. Those will allow the initial batch of data to be copied locally and then those drives will be shipped back to Gainesville as a bandwidth friendly way of getting the initial replication started. After that point only block level changes will need to pass the wire and that should keep things bandwidth-friendly.

They keep eight days of short-term retention to disk and they backup up to tape weekly on Mondays with a four-week tape retention. This is all being done primarily for Disaster Recovery (DR) purposes.

New virtual infrastructure being planned and spec'ed out

Wayne has also been kept quite busy lately designing an entire new server infrastructure. Our current SAN is three years old and the rest of the hardware is approaching four years old; much of this will come out of warranty by next September so a replacement must be considered.

Wayne is investigating moving from EMC to a new Dell/EqualLogic iSCSI SAN with a 10Gb data center infrastructure utilizing two EX4500 48-port SFP+ Juniper switches (40-ports in base, 48-port with some options). While those switches aren't cheap at ~$20k they are still an order of magnitude less expensive than the Cisco equivalent.

The idea is to virtualize everything they can. The only physical servers we would have would be the ESX host boxes for VDI/VSI, the new SQL cluster, the DNS and DHCP servers, the SCOM back-end, the file server for Dr. Borum (hipaa data and ipsec policies), the video server, and the backup servers (i.e., remote DPM server, client DPM server).

Wayne wants to divide our virtual infrastructure so that the VDI (virtual Desktop Infrastructure) is on a separate cluster from the VSI (virtual Server Infrastructure). VMware is currently running a promotion where we could potentially get an extra 150 VMware View licenses. One thought is to use the old VMware vSphere nodes to host virtual desktops for IFAS faculty and staff VIPs providing Windows 7 VMs with Office, Acrobat Reader and other basic stuff. Wayne envisions this as a general-purpose VDI for all faculty/staff under a "best effort might die" SLA--aka "trial use".

All this is dependent on budget, but now is the time to plan for this. ITSA's $202k/year budget would need to expand, but that is because most all the current infrastructure was purchased with one-time funds from various sources. Getting new and much more capable hardware on a solid recurring budget would certainly justify the additional overhead given that we need to make this sustainable in any case.

Wayne wants units to rely on ITSA rather than try to host servers/services locally. Doing that via a centralized virtualized infrastructure makes sense from both a monetary and management perspective.

Eventual move to Windows Storage Server planned

At the next iteration of our file cluster Wayne plans to move to Windows Storage Server which will provide file storage cost savings due to its implementation of single-instance storage. With that, Wayne plans to configure storage so that expansion will not involve moving large quantities of data. Currently we have an active-active file cluster with six data shares (4x8TB and 2x4TB -- 20TB per node, 40TB for the cluster). With EqualLogic and Windows Storage Server we could give every OU it owns LUN then thin-provision and extend as needed without moving data across volumes (which would also affect DPM protection groups). The new scheme would make overall management of the file cluster storage much simpler with no perceivable difference from the client perspective thanks to DFS.

New SQL cluster coming together

Wayne has been planning a SQL server consolidation to a new 2-node server 2008 R2 failover cluster for quite some time. It will support 96GB RAM in the main node and 24GB on the standby/passive node and will consolidate SQL01/02/03/04/05 and SQLDEV to a single cluster that costs less than what it consolidates and the storage allocated on the current SAN ($11,121 and $8500 for the nodes and $16,000 for the storage). This has all been on order for several months and the process is taking longer than anticipated. One server has shipped but the bigger node (the 96GB RAM one) is still in production. The storage unit arrived just today: a Dell MD3220. That is a 6Gb SAS switchless SAN with 24 x 300GB 10k SAS disks (vs 1Gb iSCSI) which really screams. Though we will have only two nodes, this unit can connect up to 8 hosts with single paths or 4 hosts w/ redundant paths.

This equipment is being obtained under a five year lease due to limited budget; doing that with this and the DPM equipment allowed getting $62k more hardware as opposed to what a 3-year lease would have provided.

ePO updates

Patch four is now available for VSE 8.7i. Wayne plans to put that to a test branch and eventually get it pushed out. Both Steve and Chris have installed it without noting issues, though Wayne said some issues have been noted via the McAfee forums. These include Mass Mailing Process exclusions not working with VSE8.7i Patch 4. There are several machines within IFAS which have had these exclusions set so they may mail-out via port 25. Also, for machines where users log on with non-admin rights, this patch apparently also causes the system to spam the security log with a ton of events. Wayne is waiting for a hotfix (due shortly) before pushing patch 4 out widely.

The protection is really based on the engine and the DATs, so patch level is somewhat less important. It basically comes down to running whatever patch has the least problems at any particular time.

There was some discussion about various McAfee woes and how to diagnose things when McAfee is suspected of causing slowness. Now that Microsoft has just released Forefront Edge Protection 2010 Release Candidate (FEP) Wayne wants to get that evaluated as a potential replacement for McAfee and its various woes and management headaches. Hopefully, FEP will prove to fit the bill.

ITSA day recordings available

In case you missed it the speakers were excellent and video of the various presentations has now been archived off the IT Security Awareness site; highly recommended viewing and much thanks to Kathy Bergsma and all the folks who helped make this possible.

Status of SharePoint services (prior discussion)

IFAS migrating to centralized MOSS

Updates as available...

Public folder file deletion policies and procedures status

Nothing further was available on this topic at this time.

Patching updates...


The November Microsoft patches included three bulletins (1 Critical and 2 Important) addressing eleven vulnerabilities--mainly in Office.

McAfee provides podcasts on the highlights of each month's offerings and another podcast summary of these patches is provided by "Security Bulletins for the regular IT guy".


A security update for Flash (version 10.10102.64) was made available last week to address another critical vulnerability.


There has been a new wave of Java exploits being used by malware. JRE6u22 was released the week after our last ICC meeting. Steve had related at a past meeting how he had managed to figure out a way to manually patch Flash remotely using PsExec. Now that Java has moved up to numero uno as a third-party attack vector, Steve decided to investigate patching that similarly:

A method for the manual remote patching of JRE

Using PsExec for remote execution

First you need to download the Sysinternals PsExec program; this is used to execute the programs on the remote machines. Steve recommends downloading the entire Sysinternals Suite. This package comes as a .ZIP file. Before expanding it you should right-click on the downloaded file, choose "Properties", and select the "Unblock" button. This permits the help files within to run. It is also very convenient to set a system path to the folder where you place these utilities. Steve uses c:\Program files\Sysinternals. You can also subscribe to the Sysinternals Blog via RSS and get notice of updates directly within Outlook.

Offline installation file

[Note: This procedure does not take into consideration the removal of JRE versions prior to Release 6 update 10; prior to that time, new versions did not replace older versions. A utility called JavaRa exists to help with removal of older versions; VBscripts are also available.]

You must download the latest offline installer, noting that there are both 32-bit and 64-bit versions.

For the purposes of this explanation, those downloads have been placed in c:\patches\JRE

For this to work you need to run a cmd prompt elevated to your IF-ADMN credentials to run your PsExec commands. You can now install/update JRE via this command:

The above command assumes your target is a 32-bit platform and the exact name of the executable will (of course) vary with each update.

Assuming the remote computer is on and reachable you should see the program get copied and run. An error code of zero indicates success.

There is lots of room for improving this as PsExec can take a list of machines from a text file, but the above does work and is a good start at handling remote patching of JAVA. You can use Wayne's "Computer info" Power Tool to determine which machines were out of compliance with regards to Java and keeping a list of machines and their patch state in a spreadsheet can assist with this manual process.


Google released a new version of its Chrome browser (version 7.0.517.44) to address a dozen vulnerabilities. Google is now paying researchers to find and report bugs in its browser. It also has an agreement with Adobe so that it can distribute the latest Flash plug-in along with its own updates. It would definitely be nice with Microsoft had similar arrangements.

MS Office News update

Some of you might enjoy this video on The Story the Office Ribbon. It is a great answer to those folks who say Microsoft changes things willy-nilly without due consideration.

Job Matrix Update status

This is here as a standing topic--no discussion this month.

Remedy system status (previous discussion)

New Help Desk software in use at Horticultural Sciences

Dennis Brown's departmental chairman had requested better reporting on IT services within Horticultural Sciences and Kamin Miller had some prior experience with Help Desk software packages which he thought might fit the bill. Investigations settled in on ManageEngine's ServiceDesk Plus. This software does a good job of combining hardware assessment with a Help Desk ticketing system. It will not integrate with our current Remedy system, however, and likely could not scale to the needs of IFAS or UF--mostly because there seems to be no way to control access in a granular and hierarchical way.

Dennis had submitted a PO for purchase unaware that there was any review process for such things at UF; nor had he considered at the time that this might be something worth discussing more widely prior to purchase. It turns out that approval of IT-related hardware and software was previously managed by Mike Conlon, but that was turned over to Elwood Aust last Spring. Elwood saw Dennis's order and forwarded it on to Dan Cromer who initially withheld approval in order to investigate further.

While Dan finally did approve the purchase, he wanted to look at this more broadly to see if IFAS couldn't agree on a unified course in the future; he also wanted unit IT to consider discussing plans for major hardware/software purchases via the ICC so that ideas could be shared prior to purchase.

Chris Leopold noted that one fairly inexpensive product that might be able to address such needs at the IFAS level was System Center Service Manager 2010.

According to Dan Cromer, UF is investigating a replacement for Remedy. Should that work out, Dennis will definitely consider moving over as his license is not cheap and they have it on an annually renewed basis.

Other Topics

Remote control solution worth consideration?

Santos Soler has been looking at some software called Enterprise Remote Control from IntelliAdmin. An unlimited license is being offered for a limited time at $599--much cheaper than other similar solutions. Unfortunately, Santos was not available to discuss this further.

Connecting to network folders on Macs

Wayne Hyde recently became aware that some folks might not know how to do this. He had sent the following which is likely a good reminder to all:

Message to the IFAS Help Desk from Wayne Hyde:
"Connecting to network folders on Macs" Wed 10/20/2010 5:17 PM

Since Mac clients do not support DFS and our login scripts out of the box, drive shares must be mapped manually and it isn’t nearly as straight forward as on a Windows client. First, you must know where the department share is located.

To find which file cluster node and share [to use] for the department, open Windows Explorer, go to \\ad.ufl.edu\ifas and right-click on an OU. Once the OU properties window appears, click on the “DFS” tab:

DFS Properties

Given this, the beginning of the path you would use on a Mac to map a folder under 4H is:


The rest of the pathname depends on which share (private, unit, users) you want, etc. The “:139” in the above path may or may not be needed depending on the version of OSX. There is a bug in older versions (10.4, 10.5, maybe even some 10.6) where it would not properly connect to Windows Server shares that are clustered. Unfortunately it is an Apple bug and we can’t do anything server-side to mitigate the issue.

The ITSA wiki documents the process at:


UAC settings egregious for users?

Mike Ryabin reported that many of his users are complaining about the enforced UAC settings. Apparently some are requesting WinXP rather than Win7 for this very reason. Mike had a last minute change of plans which did not allow him to attend today, so he asked that discussion be deferred until next month.

Wayne took this opportunity to once again preach on the evils of OU Admins adding their Gatorlink accounts as local admin on all their units' machines. That is a seriously poor practice which needs to stop. That is why we have the if-admx accounts and elevating with those on Win7 is essentially just as easy. There is no need for such practices to continue.

Web of Trust addon for IE

Steve mentioned that he is evaluating the WOT addon for IE as a potential browsing aid for his users. Steve had read somewhere that up to 20% of Google hits are pointing to potential malware risks.

PDF-Xchange (prior discussion)

Updates as available...

Interest in Wordpress blog systems, and photo gallery systems that require PHP and MySQL

Dan Cromer has asked ITSA to support faculty blogs using WordPress.

The meeting was adjourned on time at about noon.