ICC Meeting:

IFAS COMPUTER COORDINATORS
(ICC)

NOTES FROM November 13th 2009 REGULAR MEETING

A meeting of the ICC was held on Friday, November 13th, 2009 in the ICS conference room. The meeting was chaired and called to order by Steve Lasley at about 10:00 am. Steve neglected to begin recording at the start; consequently, the archived audio is missing the first 60 minutes and the notes likely suffer as consequence. Steve is looking for volunteers to remind him to "start the recording already!" each month, as that is apparently too difficult a job for him. *sigh*

PRESENT: Twenty members participated.
 
Remote participants: Tom Barnash, David Baudree, Bill Black, Lance Cozart, Dan Cromer, Nancy Johnson, Mike Ryabin, Louise Ryan, Ed Steele, A. D. Walker, and Wendy Williams.
 
On-site participants: Benjamin Beach, Micah Bolen, Dennis Brown, Andrew Carey, Wayne Hyde, Winnie Lante, Steve Lasley, James Moore, and Santos Soler.
 

STREAMING AUDIO: available here
[note: Steve failed to begin recording at the start and the first ~60 minutes is missing. Very sorry.]

NOTES:

Agendas were distributed and the sign-up sheet was passed around.

Report from the chairman

Member news:

Steve had no membership changes to relate.

Recap since last meeting:

As per his usual procedure, Steve pointed folks to the notes of the last meeting, without going into any details.

Videoconferencing and WAN discussion

[In order to make meeting participation more efficient for Patrick Pettus and James Moore, these two topics have been moved to reside the top of our agendas.]

Videoconferencing topics (previous discussion)

End-user Scheduling

Steve assumes this is still a plan in progress, but Patrick Pettus was not available to comment.

Codian bridge replacement being investigated

Steve neglected to ask Dan if any movement had occurred on this. Hopefully, we can revisit that in December.

WAN transition to CNS (previous discussion)

Updates from James Moore

James said that this year's hardware purchases are now coming in and upgrades are being planned at various sites. New improved connectivity is pending at a number of locations; Citra and Homestead are very close now and they are working on Immokalee as well.

Steve asked about wireless moving to authentication at remote sites. James said they are going into that very slowly. He believed that Lake Alfred is using that currently and that Apopka might be as well. Requiring Gatorlink accounts for all wireless access will be a new and painful process for many sites due to the need to manage the creation of guest accounts.

Policy

2009 UF IT Awards Recognition Program (previous discussion)

Steve pointed out that James Moore and Dwight Jesseman were among those recognized October 29th at the fall UF IT Assembly. Congratulations on in order for both!

ITAC-NI resurrected temporarily?

There was a meeting yesterday. The minutes should be available by the end of next week.

Course Management System Conversion to Sakai 3 (previous discussion)

Steve said he doesn't generally hear anything about the CMS Conversion to Sakai project. He suspects that Ron Thomas may have some involvement but is unsure. Steve is looking forward to see if this new system can be more palatable to our Faculty than WebCT proved to be; very few faculty in Steve's department believed the rewards to be worth the effort, unfortunately.

myuf Market (previous discussion)

Steve asked if anyone had something to relate on how this transition is going in their unit. It would appear that this system is still getting little use by end users and that most are still wondering when we will be forced to use it for most things. Steve noted that requisitions to the Software Licensing Service now go that route, but that his fiscal people apparently had not mastered attaching the paperwork via that process. Steve has received a number of calls from SLS requesting that paperwork be faxed so that orders they received can be processed. Eventually, keeping things paperless may be a boon, but the learning curve for the fiscal staff seems to be an obstacle currently.

UF Exchange Project updates (previous discussion)

General updates from the Exchange group

Steve wasn't aware of any updates to relate. There haven't been any public mentions of upcoming changes--at least that Steve had noticed. He expects that Exchange 2010 is very much on Dwight Jesseman's mind, however.

Centrally funded Enterprise CALs still on table

Steve asked Dan Cromer if there was any news on this matter. You may recall that this affects IFAS's ability to join with centralized SharePoint and affects access to a whole host of other Microsoft services. He has spoken to Joe Joyce about the matter but nothing has yet been decided.

Office Communications Server

Dan announced that he intends to fund 2000 OCS licenses for IFAS. Details of how those will be allocated are yet to be worked out and Dan was looking for suggestions. His plan currently is to offer those on a first-come-first serve basis. Access is controlled via a security group; contact Dan if you wish someone added to that.

Split DNS solution for UFAD problems

Steve wants to keep this on the agenda for future reference.

Projects

IFAS WebDAV implementation

There continues to be no progress on the documentation which was to happen prior to announcement. Since this has never been formally announced, the matter remains on the agenda as a standing item.

Windows 7 Deployment via the WAIK, MS Deployment Toolkit 2010, USMT 4.0, WDS, and SCCM

Transitioning from login scripts to GPP is on hold

Andrew Carey had previously supported moving to GPP for drive and printer mapping, but has now changed course based on closer consideration. He had previously not been concerned with the cost of SDMSoftware’s Group Policy Automation Engine, because he felt that he could complete our conversion to Group Policy Preferences in the two weeks that the fully functional evaluation software allows. Upon further reflection, he realized that we would need to continue to use the software to maintain and expand the current web based power tools (specifically the userinfo page and a planned new web app to create and permission new private folders automatically,) Consequently, Andrew emailed SDMSoftware and was informed that the software is priced per managed GPO (IFAS has 503) at $17.60 each after a 20% EDU discount for a total of $8852.00 to cover all of IFAS.

Andrew feels that Group Policy Preferences is almost definitely the way of the future, but he now questions if the product is mature enough for IFAS to make the move at this time. In a large distributed environment like ours, surrendering the ability to script the management and creation of login scripts and extract information from them except through the use of expensive third party software would be a step backwards.

Andrew has always been opposed to the login script “hacks” (http://support.microsoft.com/kb/937624, http://technet.microsoft.com/en-us/library/cc766208(WS.10).aspx - appendix A launchapp.wsf) that are available to make Vista and Windows 7 run the login scripts - but with the alternative having an almost $9000 price tag associated with it, he is now in favor of using the launchapp.wsf script instead of group policy preferences. This is the solution that has been in place for the “Login Script Test group” for some time without any issues that he is aware of and could be moved into production fairly quickly. He hopes to have this all ready by the end of the month so that Vista and Windows 7 scripts will run properly for all just as they do on WinXP.

Login script cleanup still needed

Andrew is taking this opportunity to rework the master login script code to improve on things somewhat. For example, he is redoing the printer section so that inappropriate printer mappings get deleted rather than staying around forever.

Steve suggested that it might be useful for each OU to document their individual unit-side scripts in some fashion. He has done so for his OU by creating a "Docs" folder with a text file that describes what each unit-side .vbs file does and how that all works (see here). Andrew noted that there are units that have multiple scripts, none of which point to a currently valid resource!

Andrew asked that each OU Admin take a look at those. If you need help interpreting things you can get with either Andrew or Steve for assistance, but there is no excuse for having scripts still around that point to resources which were decommissioned over three years ago.

Windows 7 deployment

Andrew Carey said that Daniel Solano has been working on setting up SCCM. The plan is to push out a thin OS-only image and deploy applications separately via SCCM. Since Andrew has been investigating this for the upcoming MPS rollout, IFAS IT is developing some expertise in its use.

Steve asked if there was anything that ICCers could do to help in this effort. There are many aspects of deployment which might be useful to explore down-the-road but Steve doesn't know how to involve people in this more widely. Micah Bolen mentioned that he has been looking at User State Migration Tool and hard-link migration in particular. So far he has not been able to get that to work. Steve noted that he is yet to find useful low-level details of how that is meant to work--for example where the files are kept and how the links may ultimately be removed.

Exit processes, NMB and permission removal (prior discussion)

Nothing further was available on this topic at this time.

Re-enabling the Windows firewall

This topic was raised by a question from Dennis Brown who wanted to know what remediation we might take against the Windows TCP/IP vulnerability which Microsoft claims is mitigated by the firewall and which they do not intend to fix on the WinXP platform. Currently, this involves a DoS-only for WinXP and not a remote exploit. Our best protection at this time is being on private IP space. Even implementing the firewall as planned will not help, because we need to pass the traffic which an attack would come in on in any case.

Limited testing of the proposed firewall GPO has been done without issues being noted. You are encouraged to test this yourself by contacting Wayne Hyde. He can provide you a way to add a subset of your machines to the scheme so you can possibly identify issue prior to broad rollout.

Micah asked what sorts of issues might be looked for. Basically, this may involve software will doesn't properly create firewall exclusions. You can see the list of exclusions via the netsh command as documented at the September ICC meeting. The GPO Wayne has created hides notifications of blocks from the user so you may need to do some investigation to determine what is going on; basically, certain applications may break if the firewall does not anticipate their needs. Should issues be located, you may add local exceptions on particular machines to get things to work; the GPO settings allow for that.

Services Documentation: Is a Wiki the way? (prior discussion)

Steve noted that Dan Cromer has been investigating getting IFAS a secure portion on the new UF IT Wiki. It appears some background work needs to occur in order to this site to be considered production-ready. Dan has requested that a project lead be appointed at the UF-level. If you are interested, you might want to subscribe to the wiki list by sending an email with "Subscribe UFIT-WIKI-L Your Name" to listserv@lists.ufl.edu.

Operations

IFAS efforts toward Green IT

Efforts at higher administration levels gaining focus

Dan Cromer mentioned having been very impressed with what Elwood Aust had accomplished for units supported by Operations Analysis regarding shutting off computers automatically on nights and weekends. It turns out that our own Santos Soler was involved in developing that project and Santos explained the process they had used.

The Operations Analysis solution

Santos said that communication with the users was a very crucial aspect. They began by asking people to shut their machines off at night and running scripts to see who did and did not comply. That helped them locate areas which might be of greater concern. The actual shutdown is done via the in-built "shutdown.exe" command on each machine.

Santos said that they made the mistake of assuming people read and followed their emails, however, and may have implemented things a bit prematurely. When the automatic shutdown was begun they received a number of complaints from individuals who had left files open and unsaved and who lost work as a result. Obviously any automatic shutdown process requires buy-in from the users in order to work.

For those needing machines to be left on for remote desktop connection, the scripts handled that by checking both that remote desktop was enabled and that the user was in the local "Remote Desktop Users" group. If so, the script would leave the machine on. Santos mentioned that he is willing to share the details with any interested parties.

Santos also said that they had a wake-on-LAN patching solution which could work around the shutdown policies and not allow updates to interfere with computer use during work hours. Such a solution, however, is complicated by the need for a machine on each subnet to handle this aspect; wake-on-LAN commands will not pass subnet boundaries under current UF configuration.

Additionally, Santos pointed out that machines fail most often when turned on. He mentioned that we need to expect that implementing a shutdown policy would lead to additional hardware issues as well. This is consistent with what they had experienced at Operations Analysis.

Recommendation request on the horizon

Dan Cromer stated that he would like the ICC to discuss this and eventually offer a recommendation via ITPAC. Because of political pressures it is likely that we will not be able to ignore this indefinitely; rather we need to address it proactively.

Most ICCers seemed to agree that there are many difficult issues to consider for this to be practical in our environment. One possibility is to focus on a specification for new machines which can support a more sophisticated solution involving sleep/hibernation with wake-on-LAN--perhaps focusing on new Windows 7 machines which better support such things.

The research environment which we support is quite different from that of a typical business office and the difficulty in implementing a power-savings plan that wouldn't cost more than it saved seems more problematic in our case. Also, the efficacy such a plan would enjoy might be considerably less than administration might hope as an end result. The trade-off between power-savings and convenience may be likened to the trade-off between security and convenience. We need to be careful that the balance is chosen wisely in a way that best supports the mission of our units. It is very easy to measure electric bills and much harder to quantify time and opportunity losses; a centralized approach may be tempted to over-emphasize the former over the latter.

Dan Cromer stated he was in favor of an opt-in program to shut down computers, stating that people who would otherwise shut down things themselves might appreciate not having to worry about that--for example, if they had left the office meaning to return and then were sidetracked for the day.

Green IT project at WEC

Tom Barnash reported via email (which was only noted after the meeting) that his department implemented a Green IT project showing how some small changes in purchasing could save a lot of money in terms of electricity usage etc. These changes didn't involve shutting down computers per se, but rather purchasing only energy efficient models of various computer equipment. Tom plans on expanding the project in the future to start looking at recyclable materials and other things related to sustainability as well.

Creating guest GatorLink accounts: singly or in bulk

Dan Cromer wanted to point out that authentication will be required via wireless at remote sites and that we should gain experience in creating guest accounts. Instructions are available for creating a single account and for creating multiple accounts in bulk. If you find you do not have the role necessary to perform that function, the instructions include the details of what role needs to be requested.

Dan seemed to indicate that careful accounting for who was using these accounts was not required; he even suggested that a single account could be shared among individuals in a pinch. It should be kept in mind, however, that it is the responsibility of the one creating the accounts to know who is using them. Steve would strongly recommend that careful accounting be kept--regardless of the overhead that entails. If we cover-up the administrative overhead then a better solution will never be investigated. It would be much better for administration to see and feel the costs involved in Steve's opinion.

Mike Ryabin asked if guest accounts could be created for durations other than 2 weeks. Currently, that is not an option. Since the account name and original password are known, however, it should be possible to change the password and thereby "remove" access should a shorter duration be desired (provided you do that before the user does themselves).

DHCP server access changes (previous discussion)

Chris Leopold's group is investigating some ”official” MS way to lock down DHCP so that an OU admin can only manage their own subnet(s). It appears that a more likely solution will be to write a web application to allow for this. Until this can be addressed, Chris asks that OU Admins forward all DHCP reservation requests to Santos, as those OU Admins on campus who previously had administrative access no longer enjoy that level of control.

Wayne said that the decision was made to restrict access because they could not risk having someone outside their group wiping out the configuration of the servers. Dennis Brown responded that perhaps it wasn't a great idea for his Horticultural Sciences unit to have given up their own DHCP server; that said, he doesn't think it is a big deal as long as he can get quick service via a request to ITSA staff. Andrew said that while Santos is primary for DHCP changes, all of them have access and can create those; if Santos is unable to respond within the needed timeframe, Andrew suggested putting out a broader request to ". IFAS-ITNS LAN Systems".

Steve asked Andrew if DHCP admin access might ever be restricted further on the MPSes. Andrew said that there may be some slight changes but that he expected them to remain as they are for the most part.

DHCP filtering to be used to exclude infected machines

Wayne noted that in the past he has disabled access for infected systems by creating a special DHCP reservation on their MAC which pointed various server settings (router, DNS, WINS) to "127.0.0.1". This solution still allowed network access within the subnet, however. Windows Server 2008 now supports DHCP filtering so Wayne can block such machines from ever getting an IP# at all. The nice thing about the former solution was that Wayne could change the host/domain name to something conspicuous like "CALL_IT_SUPPORT_YOUVE_BEEN_HACKED.LAN". That was meant to lead the user to seek support. Instead Wayne plans to have the DHCP server dump out a list of all the filtered MAC addresses. Whether that goes to a web page or to a text file on the DFS is undecided as of yet, but in any case, OU Admins will be able to view data on machines which have been filtered. Those entries will include the MAC address and a note from Wayne that provides details about why the machine was filtered.

Moving away from the IFAS VPN service (previous discussion)

Wayne Hyde said that they plan to move the numbers handed out by VPN to private IP again and implement a proxy solution for cases requiring a fixed public number. Such needs are very rare and should not dictate how things are handled for the majority.

VDI desktops as admin workstations(previous discussion)

Steve failed to ask for an update on this; he'll raise the issue again next month.

Wayne's Power Tools (previous discussion)

Steve asked if Wayne has any more tools in the works. Wayne responded that he has a couple in mind, but that it is mostly a time issue that prevents him from expanding those currently.

Folder permissioning on the IFAS file server

You are reminded to please take the time to read and implement the new standards. If you have any questions get with Wayne or Steve.

Disabling/deleting computer accounts based on computer password age

As with so many things in these times of inadequate staffing, finding time for implementation is proving difficult. Steve did want to remind folks that Andrew has a good plan for dealing with this which he simply has had no time to address.

New MPS/DC testing

Andrew Carey said that they are getting very close to deployment with the new MPS/DC on Hyper-Visor. SCCM is working for deploying a new OS to the server and then deploys the OpenManage tool on top of that. They have Virtual Machine Manager running in test mode. Before the end of the month Andrew expects to have something out at Fisheries as their test before wider deployment to our 70-some sites across the state.

Steve asked for more details and Andrew supplied them. Using SCCM, they deploy the Server Core OS onto bare metal. OpenManage is then added to support management of the hardware. When they add the physical server to Virtual Machine Manager as a host server it will install the hypervisor for you. Following that, they install the actual VMs on top of the hypervisor. It is to the point where it is pretty much automated and Andrew is fairly pleased with that aspect.

Steve asked if Andrew was involved with the DC side of things as well. He responded that he is working with Mike Kanofsky on that part. The plan is that Andrew will deploy the bare OS for the DC and Mike and the UFAD guys will actually join it to the domain and perform the DC Promo and make it a RODC. This latter part, configuring the RODCs, is the majority of what still needs to be planned.

Steve asked about how the RODCs would function. It's his understanding that the local accounts will be cached so that access to the local MPS will still work even if the outside network connection was down. On the other hand, visitors whose accounts were not part of that RODC would have to authenticate over the wire in any case and therefore might be slightly slower than they are now. Andrew said that this was pretty much how it would work, but connections are being improved all the time and over-the-wire authentication should not be an issue in general. Overall, this should be a great improvement for operations at remote sites.

Core Services status

Storage space for the file cluster

Steve noted that storage space on the file server for his department was starting to get rather low. He asked Wayne how things looked on the SAN overall. Wayne responded that upon original deployment all the LUNs were configured for 50% utilization; obviously, this has changed over time and some of those are now up to 95% utilization. As a result, then are going to buy another DAE which will add 15 drives to the SAN for allocation to the file cluster. Once that is done overall storage will go from 24TB to 40TB on the file cluster. That will drop utilization back down to about 50% from the current 80% overall.

This is a big infrastructure change that will involve a Sunday morning maintenance shutdown in order to shift everything around so the new DAE can be added--there currently isn't space at the top of the rack and everything must be shifted down to make room.

Steve asked about long-term strategies which might help with the situation: things like de-duplication or archival software. The problem is that third-party solutions are costly. Wayne pointed out that data de-duplication is not a standard feature of regular Windows Server; rather it is part of Windows Storage Server. He does hope we can eventually move in that direction, however.

Drag and drop default actions

Steve raised an issue he has seen where people have both workgroup shares (within the Private folder branch) and personal shares (within the User folder branch). When they drag files and folders from their personal share to the workgroup share, those files are "moved" and thus retain their permissioning. As a consequence, other collaborators cannot see those files. Steve wonders what the consequences might be if the default actions were changed. This would be good for such cases where the user clearly expects a copy to happen rather than a move, but Steve is unsure if there would be other unwanted consequences that make considering that option a bad idea. He would love to hear the thoughts of others on that.

Many still using the file server as "backup"

Wayne pointed out that he often sees multiple "backup" folders on the file server. Steve said that even though most of his users now redirect My Documents to the server, it remains a constant battle for him to get it out of people's heads that the server is a "place to keep backups" rather than a place which IS backed up. Steve places documentation on the user shares and creates links there to local storage points within profiles with additional documentation on which location is appropriate for which sorts of things. It has all been pretty much to no avail, however.

New version of our virtual server software available

Wayne reported that there will be some slight downtime over the holidays which he upgrades our VMWare infrastructure to vSphere 4.

ePO updates

New agent available for testing; new VSE patch, hotfix and engine being pushed

Wayne Hyde announced this via the ICC-L last Friday:

Wayne quickly added:

In summary, Windows 7 is now officially supported via our McAfee antivirus solution. The new agent can be pushed via tagging of machines within the ePO console; it is also available at \\ad.ufl.edu\ifas\security-tools\ePO-Agents. The new agent collapses the VSE and Agent beneath a new single system tray icon with some slightly better features for control and monitoring. The right-click options on this icon are richer on Windows 7 than WinXP. The status monitor has been improved:

The "Collect and Send Props", "Check New Policies" and "Enforce Policies" buttons are the items most likely of use; this will allow you to check that a machine is synchronizing with the latest ePO policies or not.

The "About" context menu item has been improved as well, providing a good deal of information for both the Agent and VSE along with a means of copying that to the clipboard:

This can be useful for relating diagnostic information to Wayne should a problem occur.

Machines showing up as non-compliant in the ePO console

Wayne noted that one possibility there involves situations where system imaging was used that omitted a crucial preparation step (ufad\if-admn credentials required). The GUIDs get regenerated on a reboot, so if you boot up your master template to makes changes you must make sure you delete the appropriate registry keys.

Reminder: Neither the Agent nor VSE are pushed out

Wayne reminded folks that it is the OU Admin's responsibility to install the agent and VSE on machines. He will make updates available via ePO, but you must take the necessary steps to get machines configured initially whenever a machine is added to UFAD (ufad\if-admn credentials required).

Status of SharePoint services (prior discussion)

Steve asked Ben Beach about the status of UF's centralized SharePoint installation. Ben replied that they are in the process of getting the DNS reservations worked out with ufl.edu. Steve asked if IFAS would be involved and Ben replied that Dan Cromer wants to merge our sites with UF's. CALs are the issue there, however, as we run the free SharePoint Server version and UF plans to implement the full MOSS.

Public folder file deletion policies and procedures status

Nothing further was available on this topic at this time.

Patching updates...

Microsoft

The November Microsoft patches included three critical and three important updates for Windows. A podcast summary of these patches is provided by "Security Bulletins for the regular IT guy". You might also be interested in knowing that Microsoft publishes a bi-annual Security Intelligence Report that is useful in monitoring the changing landscape of security threats.

Fighting malware

Wayne stated that most malware is installed by users inadvertently via their interaction with the system. Many runs with local admin rights and thus any "nasties" they encounter have full system access. Malware has become so sophisticated that signature-based detection is essentially useless. Heuristic detection is promising, but our antivirus solution is not particularly strong in that area currently. Wayne speculated that the most significant step we could take to protect things would be to get people to run w/o admin rights. While Steve agrees, he pointed out that standard user malware is just waiting to take over the lead in that case; the problem will never go away no matter what actions we take. All we can do is try to stay ahead of the game as much as possible.

Drive-by web-based pop-ups which attempt to impersonate "system events" and thus lead the user to initiate installation are now a common thing. These ploys trick people into clicking and installing many versions of fake "antivirus" malware. The thing to do should you run into such a thing is go to the task manager (Ctrl-Shift-Esc) and delete IExplorer to kill the browser process. The "dialog boxes" these things throw up are phony and clicking anywhere on them, including the pretend "close" or "cancel" buttons will only allow them to proceed; rather you can press Alt-F4 to close the window in a way that the program cannot trap. Killing Internet Explorer is your best bet, however, to avoid problems should a pop-up arise.

Issue at UPD

Santos reported that UPD recently had to wipe and reinstall all their computers due to malware infections. They were all running as local admin and their antivirus and other updates were not being attended to apparently.

Applocker for Windows 7

Micah asked about the feasibility of using Applocker. The problem there would be maintenance and user acceptance, as we run a wide variety of software in our environment.

Emailed notices of VSE detection events

Wayne plans to investigate the reporting capabilities of ePO to see if he can get it to send out reports to OU Admins regarding the most serious of detection events. Those generally involve detections within the Windows system directory and may be easily viewed already in the "Detections in \Windows" panel of the Detections Dashboards in the ePO console.

MS Office News update

Steve let everyone know that he updated the Office 2007 deployment site and it should now be current through November 2009.

Job Matrix Update status

This is here as a standing topic--no discussion this month.

Remedy system status

Steve wants to leave this matter as a standing agenda item for future discussion.

Other Topics

Removal of redirects for consolidated web sites (prior discussion)

Steve forgot to mention this at the last meeting, but had added it into the notes after the fact.

SMB v2 re-enabled?

SMB 2 had been disabled on our file servers due to a remote code execution vulnerability. Steve meant to ask if that has been reversed now but forgot.

PDF-Xchange (prior discussion)

We did not discuss the matter this month, but Steve would like to leave it on the agenda.

The meeting was adjourned just a bit late at about 12:10 PM.