IFAS COMPUTER COORDINATORS
Message from Wayne Hyde to the Accordent-L list:
There are a few issues which may affect the Accordent machines:
The WSUS updates should not be a problem. I am working on a new ePO tag ‘ACCORDENT’ which will allow OU admins to apply a different update/scan schedule to the Accordent computers.
How many people are having technical issues with the audio dropouts that may be due to McAfee?
Please include me in the CC list for discussions on this topic as I am not a member of the ACCORDENT-L list.
Other than Ron's mention, nobody was sure of exactly who was experiencing this issue. Apparently, no one has responded to Wayne on this.
Webinars on Lecture Capture Systems from CCUMC
Steve recently found some interesting seminars from CCUMC comparing various lecture capture systems. These include a presentation on UF's MediaSite usage (along with Echo360, Panopto CourseCast, and Opencast Matterhorn). There is also one covering UB's Accordent usage (along with Tegrity and Camtasia Relay).
WAN transition to CNS (previous discussion)
Updates from James Moore
James did not attend today.
Dan Cromer had sent out the following:
Message to the ICC-L from Dan Cromer:
I had intended for some time, at the request of CNS to develop a survey for Wi-Fi across IFAS non-wall-plate and non-Gainesville units. However, I think it will be easier for you as a unit IT support person to send an e-mail rather than go to SharePoint or other survey site, so here's the urgent request. Please send a list of the current and desired wireless access point in your unit. Mailto: firstname.lastname@example.org, with a copy to me. This list will be used to develop an upgrade and replacement plan. Include equipment vendor(s) and quantity. Thanks.
Steve asked Dan if he had gotten sufficient feedback on this. Dan responded that he would appreciate more feedback still and Francis Ferguson said he would be submitting his information on Monday.
Dan said that Lake Alfred was in progress and that he believed Immokalee was next. After that it will be a matter of first come first served.
Donna McCraw had shared that the next Peer 2 Peer event has been set for Friday, March 18, 2011 from 9 AM - noon in Smathers 1A. Elias Eldayrie will give an update on the IT governance structure. There will be an official announcement later when the speaker list is finalized.
Upcoming ITPAC meeting
Another ITPAC meeting is scheduled for February 28th at 1:30 PM in McCarty D room 1031A. Currently no agenda has been published.
Dan Cromer asked if the ICC agreed with the migration plan as discussed by Santos Soler because he wanted Dennis to be able to relate that support at the upcoming ITPAC. While Chris Leopold didn't see this as a policy matter but rather a point of information, Dan disagreed saying that we need to have administrative support behind us on this. If pushback occurs during implementation Dan wants to be able to refer to the ICC's and ITPAC's support on the matter to provide credibility. Steve responded that, having heard nothing to the contrary, Dan could assume that this had our backing. Steve then urged anyone with a dissenting opinion to please make their position known as soon as possible.
CNS wants to host DHCP/DNS solution for all campus
Dan Miller has notified Chris Leopold and Dan Cromer that CNS is deploying new BlueCat servers (like HSC but higher capacity) that will provide site redundant DNS and DHCP. They support secure DNS, windows DDNS, and delegation of control for updates for DNS and DHCP.
CNS hopes they can get all of campus to use these new services once they are ready and they may or may not want to wait for campus IPAM.
Dan Cromer had responded that he would welcome and support central UF DHCP and DNS systems as long as shared delegation of control is available. Dan Cromer had since heard that CNS was very much interested in delegation as they did not have the staff to handle this in any case.
Steve noted that we all have good access to our DHCP logs now (if-admn credentials required); as long as a similar system was in place there shouldn't be any big issues at the unit level.
Chris Leopold mentioned that his only real concern related to knowing what level of integration they were looking at. He wondered if they were looking just at the ufl.edu fully qualified domain name or if they would also include ad.ufl.edu. The latter would raise questions of integration with the WAN, etc. Basically, Chris thinks that a great deal of technical investigation will need to precede any change. Chris had raised this to Iain's attention and suggested he contact Dan Miller in order to get into the discussion as well. Chris suggested that DNS would likely be a bigger issue than DHCP in any case, but we will just have to see how things transpire.
Dennis Brown had already gone through losing direct DHCP control to IFAS but said that Santos Soler was always very responsive and things were going well. If that level of support could continue then he would have no problems, but if a ticket had to be submitted in every case, he felt it might be a step backwards.
Mike Ryabin asked if this all meant that every time he needed to make a DHCP reservation that he would need to go through CNS. Steve responded that it was his understanding this would just be for campus in any case, but Chris Leopold wasn't sure on that aspect--after all, the IFAS remote sites are still on the ufl.edu network. Dan Cromer added that our requirement for delegation should mean that our server administrators would have the control we needed to continue making changes as we do currently.
Lync vs. Cisco Unified Communications (previous discussion)
UF is already licensed for the Enterprise Voice part of Lync and just needs a SIP trunk for Lync to become a full enterprise phone system. Add a $60 headset and anywhere you run Lync, you have your office phone. This is something Luis Molina had at his previous school and it includes very cool extras like voice mail in your inbox, reading your email to you over your cell phone, etc.
A SIP trunk apparently has already been approved for our OCS setup and Microsoft is offering up to $25K to assist us with a pilot for initial install/overview of Lync; this would interconnect that with our phone system.
On the other hand, there have been rumors that CNS is well into an unpublicized pilot of Cisco Unified Communications--a direction that HSC has already adopted. The question is whether UF can or will support both the Cisco and Microsoft solutions for unified communications. Cisco would appear to have the upper hand with those in charge at CNS.
Here are some case study links offered by Joe Gasper from a recent thread on the Windows-Hied list:
Steve asked Dan Cromer if he thought this was a case of competing systems or if those two could both coexist. Dan responded that the Cisco softphone client is available right now for around $80 and that will permit you to use your computer as your phone; this software talks to the Cisco Callmanager directly. Dan saw the SIP trunks demonstrated on Wednesday by Luis Molina, where Luis called his cell phone using Office Communicator.
Dan had a conversation with Iain Moffat recently where they both agreed that paying ~$80 times 2000 (IFAS) for the Cisco software didn't make much sense when MOC is already available at no extra cost. That's not even considering the fact that MOC/Lync naturally has better integration with our other Microsoft products and has an available web interface as well. Why would IFAS want to pay $160,000 to lose features? Dan Cromer added that he thought Elias was of a similar mind on this.
Mike Ryabin noted that he learned that Accordent is now offering integration with Lync and SharePoint. This will allow the recording and publishing of Lync sessions via Accordent among other things (see also). Mike said he would provide Dan with the Microsoft contact he had regarding this in case IFAS wished to look into that further.
Lync being demoed at VoiceCon in Orlando
Mike Kanofsky's position has been posted
This posting doesn't seem to put much emphasis on Microsoft solutions expertise--which was what Mike had in spades. Steve may not understand how these things are supposed to work but it is somewhat troubling and surprising to him considering what Iain had led us to believe at last month's meeting. It seems odd our new “Microsoft” guy should have “Preferred Qualifications” in the area of “open systems, VMWare Infrastructure, virtual desktop infrastructure, Unix Infrastructure, Oracle, Bash and Perl” but no preferred Microsoft certifications or credentials. It was also concerning that the current job posting appears to be a scant 7 business days and nothing had been posted yet to the email@example.com and other lists as of last Friday.
A number of UF folks queried Iain about this and here is his reply:
Message from Iain Moffat:
Here is the PD as it was sent to HR.
I think it is pretty clear on duties importance, etc...
Yes, I agree that the posting could have been clearer. It looks like certain components were taken from the PD, but not in the priority that one might expect.
As to advertising, etc... Tim decided to post locally. He feels that there are good local candidates that are interested and he wants to get someone officially on-board quickly. We do have internal candidates and others interested, so we will have to see what comes out of this first round.
As I have said openly in previous meetings. If we do find that an internal candidate that is a good fit, we will not be down a position. If that happens, we will backfill that position after updating it to reflect appropriate duties: AD, Exchange, OCS, SQL etc... as needed by the team overall depending on current skill-set and needs.
I hope this helps a bit to clarify things.
At our last meeting Iain had mentioned his plan to advertize this widely, but it appears that Tim Fitzpatrick vetoed that and they are looking locally only. Steve believes that this position would attract widespread talent if given the chance. Even given what transpired Steve finds it odd that this position wasn't mentioned on any UF lists like Activedir-L or the CCC.
Steve had suggested to Chris Leopold that he ask if IFAS might have Andrew Carey sit in on the interviews. Having outside members on interview committees is something which Chris Leopold has done in the past (e.g., Kathy Bergsma with replacing John Sawyer) and which Steve believes benefits all. Iain has not responded to that yet, but Steve hopes he would see this as an opportunity rather than an imposition. While Steve realizes that CNS has every right to handle their hires as they see fit, he hopes they also realize that other parties at UF are greatly affected by such decisions and might have constructive input if provided the chance. Steve is convinced we all have the same goals in mind.
Centralized FAX service via Exchange (previous discussion)
Updates not available...
Uploading multiple resources via WebDAV with Windows 7
There have been numerous difficulties reported getting WebDAV to work with Windows 7. One solution is to install a free third-party client from jscape called Any Client. That said, Steve was finally able to get this to work with Windows 7 via instructions within the "Create a WebDav on a Windows 7 machine" section of the online resources.
IT survey is coming (previous discussion)
Updates not available...
Outsourcing of DE course development (previous discussion)
Updates not available...
Updates not available...
Steve wants to keep this on our agendas in case discussion seems warranted.
Steve wants to keep this on the agenda for future reference.
New web cluster
Santos sent out the following to all his site owners who had FrontPage Server Extensions enabled on their web sites:
Message from Santos Soler:
You are receiving this email because you have or had a website on an IFAS web server. If you have a site on our server(s) there is a lot of information below that may affect your site - PLEASE take some time to carefully read the information below. If you do NOT have a site please contact me, and I will remove you from this mailing list ASAP.
I will be moving all IFAS web sites to a new web server. This new server will provide improved security, stability, and will have many new features.
The move to the new server means there will be some small changes. FrontPage Server Extension has been retired by Microsoft; it cannot be used in the new web server. This affects a number of sites on our old server. You are receiving this e-mail because one of the site(s) you maintain uses FrontPage Extension. Please check if you have any web forms on your site.
To prepare for this move we will need you to do some “cleanup” of your web site. Please remove outdated content, backups of sites from previous years, content that does not adhere to UF’s Acceptable Use Policy, etc. I will look for folders named backup, old, copy, etc. before moving them to our new server. We also host a separate video server for IFAS. For improved efficiency of media streaming, we will be moving video content to this server, including WMV and WMA files.
NOTE: This process is going to be manual and I will be reviewing content and sites for outdated information before moving it to the new server. If your site has not been updated in the past year you will get an email asking if it should remain active, to comply with IFAS web policies. We will be working to get all sites into compliance with IFAS web policy during this move to the new server.
Below are links to UF and IFAS web policies.
Please let me know if you have any questions.
Santos said that he had gotten a flurry of emails asking if people could still edit web sites with FrontPage. Santos said that they certainly could, but that some of the features would have to be avoided--namely those forms features which depend on FrontPage Server Extensions (FPSE).
Steve asked if we had a "canned" forms solution to offer folks as a substitute. Santos responded that he has been referring people to Ligia Ortega who apparently has a php-based solution for that which can be adapted to most instances. This does require a bit of coding expertise, however. In any case, Santos has painstakingly determined who might be affected and has contacted them about the need for fixing/cleaning up any such FSE dependencies.
Another issue entails sites which use IIS passwords for access. The feature being used currently will have to be replaced when migrating to IIS 7.5. The issue is that the current passwords are encrypted; Santos either will have to contact each site to try and find what the password is currently or set a new password and inform folks of that. This is his second major task which requires coordination with site owners.
The final major step will involve the actual moving of sites. That process must include cleanup of unneeded files (many have backed up old copies of sites online for example), modification/consolidation for compliance with IFAS and UF web policies, and the transfer of media files from the web server to the media server.
Santos rehashed the various reasons that compliance is necessary for our web server to be maintainable and for our web presence to better support our mission. All of those reasons and more are available from the historical documentation following how these IFAS policies were developed. Some of this information is available from:
Accordent recording space running out?
Mike Ryabin said that it appeared that space was already growing short and asked if that was a concern. Santos said that space should not be an issue. The video portion of each recorded session goes to the media server and there is plenty of space there. Santos said that these lecture captures use much less disk space on the web server than on the media server; that, in combination with our move to a new web server means space should not be a problem.
Winnie Lante looked at the web server space being used by her Accordent and was surprised to find her 66 recordings were taking up over 15 GB. Steve notes that his system currently has recorded 39 lectures that are taking up 1.21 GB on the web server and 6.7 GB on the media server, so Winnie's configuration is using considerably more space per recording on average; that might possibly be due to their length.
Santos reminded folks that some reconfiguration of each Accordent system will be needed when migration to the new server occurs. Since our Windows Media Server only serves WMV, WMA, and MP3 file, Santos also has plans to create a second server to handle alternate content such as mpeg, mp4, flash, etc. It is his hope that this will allow even further cleanup of the web server via relocation of various media files.
On January 25th Andrew Carey announced: "The last of the original 52 UFAD DCs running Windows Server 2003 in the IFAS remote offices has been removed from service and replaced with Windows 2008 R2 RODCs. We still have 17 sites which did not previously have DCs that will be receiving new servers once we receive additional server enclosures, but I wanted to thank everyone for their work on this project so far." Now Andrew can turn his attention to getting the Multi-Purpose Servers (MPS) going.
Andrew commented that the MPS work has indeed begun. There were a few that had already moved due to hardware failures, lack of disk space, etc. Triage is continuing so that sites with the greatest needs hopefully will be addressed first. Andrew is working on IRREC currently because they have been relying on a separate aged 8-year-old non-MPS box. Chris Leopold has just about wrapped up the Putnam CEO.
Sites getting MPS servers for the first time are quite easy to handle and Andrew can rock right through those upon request; it is just a matter of coordinating dates with CNS. The existing MPS servers will take longer though and there are many of them. Again, most of the CEOs will not be too bad because they are fairly small; it is the RECs that will take considerably longer.
Chris Leopold added that they have received twenty new server enclosures which he and Ben Beach will be preparing for distribution to Districts 1, 2 and 4. Those enclosures are the only real holdup on the deployment of the remaining small set of server boxes; the servers have actually been deployed sans cabinets in the meantime to address the DC platform upgrade issue.
Immokalee noting extremely long logon times
Kevin Hill reported that since they installed the new DC/MPS there they have noted that fewer than half of the local machines are using the local RODC as the logon server. Even those that do seem to generate a lot of traffic back to Gainesville over their currently very crowded link. Login scripts have taken five minutes to run, for example, and thus network drives aren't showing up until much later than people expect.
Kevin wondered if we might have some incorrect DNS entries related to sites. He has been unable to diagnose fully but things certainly are not working correctly. Andrew responded that Kevin is the only one who has reported such an issue currently, but his investigation had found a probable cause. Andrew explained that in order to authenticate to a RODC both the user and the computer must be members of an allowed group. We have existing autogroups for handling the user side, but Andrew had to write a new script to populate machines to their appropriate groups. Investigating, Andrew found that this script was maxing out on the number of objects it could add at a single time. He is working on a fix for that by creating individual groups for each OU; he hopes to have that corrected by early next week.
More investigation may be needed, but Andrew hopes that script change will improve Kevin's situation. Kevin asked if a computer shouldn't use the local netlogon share once it authenticates--even if the original authentication had to go over the wire for some reason. The five minute delay is what is concerning Kevin and he is having trouble understanding that aspect. Andrew said he believed that would happen only if the RODC was caching the credentials. Kevin asked Andrew to let him know when the script changes were complete so that he could run additional tests and re-evaluate.
Chris Leopold noted that DFS is not implemented at our remote sites currently but they are considering implementing that and changing the login scripts to point to our \\ad.ufl.edu\ifas namespace. Chris wanted folks to be aware and to let him know if they anticipated any issues with that.
Wayne Hyde said that the new SQL cluster had been in production for some time, but they are still trying to migrate databases. Matt Wilson has about three more to move off one box and then they are also waiting on the SharePoint migration to free up another SQL box here. The cluster is working well, however and obviously is a lot faster.
Both our current virtual infrastructure and our file cluster will be migrated to the new hardware. The file server storage structure will be reworked as well. Previously our best option was to have multiple departments share LUNs; in the future Wayne intends to provide and individual LUN for each department which can then be expanded in-place as needed. Right now the creation of more space involves transferring data between LUNs which is a slow and tedious process. In the future we will be able to take advantage of thin-provisioning and then adjust storage as needed basically on-demand.
These changes will involve changing of some cluster names, unfortunately. That will be transparent to Windows clients because they handle DFS; Macintosh and Linux clients however will have to manually adjust to the new paths. Downtime for the transition should be minimal because we will have both the old and new storage to work with. The data will be robocopied over, access will be denied to the old share, and the DFS namespace will then be changed to the new server. Windows clients should then automatically reconnect to the new location.
There continues to be no progress on the documentation which was to happen prior to announcement. Since this has never been formally announced, the matter remains on the agenda as a standing item.
W2K8 R2 and Win7 SP1 released to manufacturing
W2K8 R2 and Win7 SP1 RTM'd Wednesday. Windows Server 2008 R2 will finally be made available to download via the web on Tuesday Feb 22nd, following a release to MSDN TechNet customers on the 16th.
Steve warned folks that there is a known issue with the RSAT tools. One will receive a "The update does not apply to your system" error when attempting to install the tools on integrated SP1 clean installs. A re-release of the tools is planned for April to deal with that issue.
UF SCCM Support Group
Steve noted seeing on the SCCM list a mention that Adobe has finally released official SCUP Catalogs for Flash, Reader and Acrobat. He asked Andrew if he knew anything about these, wondering if application updating might be an easier issue to address first with SCCM rather than full-blown OS deployment.
Andrew responded that this was a method by which vendors could package updates for deployment via SCCM. Andrew added that Nick Smith is working on SCCM and he believed he was close to having a patching solution for Java. Steve thinks the patching of 3rd-party applications via SCCM would provide a huge return on time invested if Nick can manage that.
USMT 4.0 update
An update has been released for the User State Migration tool which includes support for Office 2010.
Updates not available...
Re-enabling the Windows firewall (prior discussion)
Updates not available...
Services Documentation: Is a Wiki the way? (prior discussion)
Updates not available...
Updating the IFAS software download site
The IFAS software download site (if-admn credentials required) lost links to many of the packages offered when Erik Schmidt removed the vista.ad.ufl.edu download site earlier this week. Santos Soler had originally asked for ICC discussion to learn if some packages, such as Office 2003, could be retired. In the meantime, however, he has already restored everything that was there prior with the exception of Vista. Should anyone require that, please let him know.
Restoration of back-ups on the file server
Wayne Hyde intends to document and announce proper usage as time permits.
Membership of ". IFAS-ICC" email distribution group to be narrowed to ICC members only (previous discussion)
Steve will keep this as a standing item on our agendas for now as a reminder. The ICC distribution list is more targeted and restricted to IFAS IT support folks only.
Steve had left this on the agenda in case further discussion was deemed warranted.
Steve wants to keep this topic on our radar.
Moving away from the IFAS VPN service (previous discussion)
Cisco Anyconnect client
In the meantime Steve has been recommending the built-in L2TP/IPSEC client configuration for his users as a first option. Steve prefers not having to install a special client if that is not needed. Now, however, CNS has a new generation of Cisco client, the Cisco Anyconnect Client which they believe is superior to past offerings due to its ability to build reliable tunnels in "difficult"network environments, as well as the ease of installation, configuration and updates.
Chris Griffin explained:
Message from Chris Griffin to the CCC list:
We have actually done a fair bit of throughput testing in our lab on a wide variety of OS and clients during the beta period and have found the Anyconnect client either performed the same or better than the L2TP or IPsec clients in all the tests we ran. Just went and looked up mac testing. Macbook Pro "6,2" (2.53Ghz processor). Connected to a 100Mbps switchport without Anyconnect client and iperf. Bidirectional throughput = 93.6Mbps. With Anyconnect client = 86.4Mbps via SSL. Additional CPU load was 6%. The results are similar for an older macbook pro and a mac mini that we also used in testing (although the load was a bit higher on the older boxes). When testing over cable modem, the client was able basically max out the 12MBps down/1.5Mbps CM up without issue. Effective throughput was around 10Mbps down due to the VPN overhead.
With the new client, it uses SSL over TCP to set up the signaling and also brings up a DTLS tunnel which is UDP based and is used for most of the traffic. If, for whatever reason, it can't bring up or use the DTLS tunnel, it will use classical SSL to encap/decap the traffic. This is TCP based, which means if the TCP stack isn't tuned properly, you might get slower than expected performance. MacOS tends to start with small TX/RX buffers, but does some adaptive tuning. Some TCP tuning might be in order to get maximal throughput if you have a good bit of latency in the path. We do see a higher instance of Mac clients not using DTLS tunnels and we are investigating this with our vendor. We will let you know what we find, and if there is a client issue, we will push a fix when available.
Dan Cromer had mentioned noticing that http://vpn.ufl.edu was not available as a redirect to https://vpn.ufl.edu. Chris Griffin agreed it would be much more user friendly to have the http -> https redirect but that there was a technical issue currently. This site is actually hosted on the VPN equipment itself and they currently support using port 80 with the legacy IPsec client; consequently, that port is already in use for something else. Chris added that they are looking at removing support for port TCP/80 encapsulation for the older client so they can turn on redirects for the Anyconnect installation process and said they would keep the list updated on that status.
Dan added that he believed this new client was easier to install (for those having admin access of course) that it would be to configure the L2TP/IPSEC client. Most seemed to prefer native configuration over installation (particularly of a Cisco product which historically have caused issues). Installations continually have to be updated and patched. CNS had mentioned wanting to retire the L2TP access eventually, but that is deemed to be very far into the future still.
Steve asked Chris if this "port 80 issue" had anything to do with CNS wanting to retire L2TP/IPSEC support. Chris responded that port 80 related to IPSEC but that L2TP requires Generic Routing Encapsulation protocol support which should be but is not always enabled on routers. Consequently, a L2TP connection cannot be built in a small minority of circumstances (basically what amounts to a firewall issue). L2TP is thus viewed by CNS as potentially less reliable.
Dan Cromer reiterated that he has been trying to get ITSA to move our VPN to private IP for a year now. Chris Leopold responded that Wayne too has been pushing for that and it is still awaiting sufficient spare time for implementation.
VDI desktops as admin workstations (previous discussion)
Wayne announced that he now has a small pool of Windows 7 VMs that are set up as administrative workstations with the RSAT, etc.
These might be of use in certain instances as he noted a new peculiarity relating to restoring "previous versions" of files from our campus file server. Should you try to restore files for a user via your if-admn account you will most likely encounter an error due to the fact that the shares are on the file server while the shadow copies are on DPM. When switching between those two contexts during a restore attempt, another instance of Explorer gets launched, but with your GatorLink credentials and access is denied. One solution is to teach the user to restore their own files (preferred solution if possible) or do that for them either on their box using their credentials over via an MOC/Lync shared desktop session. The Win7 VDI admin desktops will permit logging on as if-admn as another potential option.
Steve noted that these VDI admin workstations require the VMware View client for access and that one of Tuesday's MS patches breaks the former clients. Wayne added that the corrected client unfortunately uses the same build number so that when one tries to upgrade they get a notification that this version is already installed. The fix there is to uninstall and then reinstall; unfortunately, reboots are needed at each step as well.
Wayne announced a new Power Tool to assist with the recent Java Runtime Environment Exploits initiative announced by Rob Smith:
Message from Wayne Hyde to the ICC-L:
I have created a new WPT that will query the Lansweeper database and output a report of software on your OU computers given various search criteria.
Click the “OU Computer Software” link after logging in with your management account.
A sample report searching for “java%” using the combined view (one table for all found software):
Ignore the trailing “\1” at the end of the computer name. Lansweeper uses the postfix to designate unique computer names. If there are more than one computer with the same name in the database the first will have “\1” and the second “\2” and so on. I’ll work on making the output pretty when I’m not on “vacation.”
A sample report searching for “java%” using the separate view (each computer is listed in a separate table):
You will note that multiple versions may be listed – Lansweeper queries the registry for a list of installed software so it is possible to see multiple versions of an application. The list reported by my tool is what would be listed if you go to add/remove programs in the control panel.
Quick and dirty, but it will get the job done. I will also make it possible to change the primary sort field so you can sort by Software Name instead of Computer Name, etc.
Steve noted that “J2SE%” would be a good term to run with the Software filter as well as that will pick up vulnerable installations as the result of SAS installs. The "%" is basically a wildcard which will match any number of characters within a given position within the string; one may use multiple "%" symbols in a filter.
Computer compliance tool in production (previous discussion)
Chris Leopold asked for feedback on adding another tool which would report user folder names matching user accounts that were no longer within your OU. His point was that might help with cleanup. All agreed that this would be another useful reminder for things which may have fallen through the cracks--i.e., NMB was removed but files were not decommissioned.
You are reminded to please take the time to read and implement the new standards. If you have any questions get with Wayne or Steve.
This is yet another matter for which finding time for implementation is proving difficult. Steve wants folks to remember that Andrew Carey has a good plan for dealing with this which he simply has had no time to address. In the meantime, it would be very good of each OU Admin to consider mimicking the proposed plan manually by keeping their own records and deleting any computer object which have been disabled for 90 or more days; Wayne's Power Tools can identify those. Steve has finally begun doing that for his own unit and it has made his view within ADUC much more agreeable.
Core Services status (previous discussion)
IFAS file server issues
Back in late January we had problems with the file servers. Wayne Hyde explained as follows:
Message from Wayne Hyde to the ICC-L:
The PowerPath upgrade from 5.3SP1 to 5.5 on CNFILE2 failed yesterday morning causing the node to crash on boot and then after recovery it could not see any SAN LUNs. Long story short is that CNFILE2 was evicted from the failover cluster and CNFILE3 was built to replace it for the time being and will eventually remain as a third node. We are waiting on log analysis from Tier2 support to know if CNFILE2 can be resurrected or if it needs to be rebuilt. A quote from Aliens that I usually state in regards to malware infections comes to mind….
All of the cluster file server resources are currently hosted on CNFILE1. This weekend (either late Friday or early Saturday) we will be testing failover of the cluster resources to CNFILE3. If all goes as planned, the file shares will only go offline for roughly 30 seconds for each of the two cluster resources.
If you experience any problems or have reports of users not able to access the file shares, please contact Andrew, Chris or myself immediately. We had a false alarm or two yesterday, so here are some quick and easy steps to check if you can reach the file cluster which may eliminate server-side problems.
Open up a command prompt and type the following commands:
Ping if-srvc-file Ping if-srvc-file1 Ping if-srvc-file2 Ping if-srvcn-file1 Ping if-srvcn-file3
You should get ping replies from all of the cluster names (first 3 listed) and physical node names (last 2).
Net view \\if-srvc-file1 Net view \\if-srvc-file2
The two “net view” commands should return a list of shares such as the following:
C:\>net view \\if-srvc-file1 Shared resources at \\if-srvc-file1 Share name Type Used as Comment ------------------------------------ DATA-K Disk DATA-L Disk DATA-M Disk The command completed successfully.
You can also test if your unit’s DFS path is working by:
You should see a directory list including your private, unit, and users folder. If you only see a directory listing with the OU name, we’ve got problems.
Wayne reported that these issues should be past us now. File3 was built and added to the cluster so that we now have file1 and file3 currently. He will rebuild file2 and add it back into the cluster tonight. Then he will kick file1 out and rebuild that. The end consequence is that we will have a 3-node cluster instead of a 2-node cluster.
A new version of VSE is available that hopefully will improve performance issues.
Message from Wayne Hyde to the ICC-L:
McAfee released VSE 8.8 yesterday. The components have been checked into ePO and the ePO Policy Migration tool appears to have worked for once. The 8.7i policies and client tasks have been migrated, but I have not verified all of the settings. With that said, VSE 8.8 is available to deploy on select test machines at this time. After a week of testing and positive feedback from you, I’ll set 8.8 as the default AV install in ePO using the PushAV tag.
To deploy VSE 8.8, you can either install manually or using the Push88 tag in ePO. The client binaries for manual install are available on the SECURITY-TOOLS share:
The Push88 tag activates a client task (the CMA must be installed for this to work) that attempts to install 8.8 immediately after an agent check-in or wake-up call and at every policy enforcement interval. The steps you should take to push a test install are as follows:
If you are doing this to your local machine you can use the McAfee Agent Status Monitor to view the progress. The Push88 tag will be cleared after installation about an hour via an ePO server task.
Let me know if you have any problems.
Here are links to various documentation files:
Wayne had noted one bug documented only in the beta1 release notes where including a UNC share in the environmental PATH causes a failure of the VSE services to start. No one anticipated this as being a problem in our situation, however.
For the most part reports have been favorable with improved scan performance noted--perhaps due to the caching now used. Steve's own early praise has wilted somewhat since Wayne discovered that the cache is apparently torn down with each DAT release. This would mean that performance improvements will only be observed should one do multiple scans on a given day...which is rather unlikely. Wayne thinks McAfee was basically just gaming the benchmark with this. There are other performance enhancements, however, so he is hoping this will still be better overall.
Steve noted that the McAfee engine hangs with .jar files that occurred regularly prior on his systems seem to have vanished with this new version--which is a blessing.
Wayne is pushing ePO CMA 4.5P2 out this week [Build 18.104.22.1680--right-click on system tray icon and select "About..." to verify]. The task updates the agent at ~ 5AM or so and it will run the missed task after a startup w/ 10 min delay. Currently about two-thirds of our machines have updated. Wayne is creating a list of roughly 100 or so systems with older agents that aren't updating which he will share with us. He also has lots of non-UFAD machines in the "lost&found" group; it appears they were in UFAD at one point and have been removed, so he has to do some house cleaning.
Status of SharePoint services (prior discussion)
IFAS migrating to centralized MOSS
Steve mentioned that he had sat in on the OSG meeting Monday via Live Meeting and that Chris Leopold had as well. It sounded like Buck is getting close to considering an IFAS migration. They were talking about interim licensing issues because we must first upgrade before migration apparently. It also seemed to Steve that they almost were making fun of the way we were doing things--something Steve did not understand.
Chris Leopold said that Dan Cromer arranged to buy some migration software for about $6K. Chris felt that this payment should entitle us to some support in getting things migrated, but apparently CNS is of a different mindset. Chris noted he doesn't really have the staff to pull this off and suggested that $6K might be spent on hiring someone to migrate us instead. Details of how this migration will take place and particularly the details of what features we can and will have once migrated are still very much unresolved. This is getting to the point where we need to move on it, however.
Dennis Brown asked Chris how much of their time might be freed-up once this move was complete. Chris responded that it actually would be very little as our current system has been fairly stable; design and content details will remain with us. Steve pointed out that the main savings might come down-the-road when future upgrades occurred which would presumably be handled for us.
Nothing further was available on this topic at this time.
Has UAC saved us from rogue antivirus compromises on Win7?
Wayne was curious about our take on this question as he had noticed such UFIRT alerts as having occurred only on Windows XP where the user was a local admin on the box. Steve responded that he had a user that was saved this heartache just yesterday, so he assumed that wasn't the single occurrence.
Cisco 2010 Annual Security Report
Steve recommended the Cisco 2010 Annual Security Report as being a good read. Among other things, it points out that cybercrime is moving toward mobile devices. Life is bound to get rough for those soon.
The February Microsoft patches included twelve bulletins (three "Critical" and nine "Important") covering 22 vulnerabilities affecting IE and various versions of Windows as well as Visio.
McAfee provides podcasts on the highlights of each month's offerings and another podcast summary of these patches is provided by "Security Bulletins for the regular IT guy".
Dan Cromer had forwarded the following:
Message to UF Campus IT Directors from Rob Adams
One of the most common exploits seen by the Information Security and Compliance Office is an attempt to compromise a system via flaws in older versions of the Java Runtime Environment (JRE). These types of exploits typically cause the clients JRE to attempt to download a malicious file which is detected by the UF Intrusion Detection System (IDS).
To reduce the number of JRE compromises on campus, the Information Security and Compliance Office will initiate two new programs as follows:
A separate email has been sent to "NET-MANAGERS-L@LISTS.UFL.EDU" providing the same information.
If you have any questions, please direct them to Jim Hranicky.
Jim Hranicky's more technical post had preceded the above message:
Message to Net-Manager-L from Jim Hranicky
Starting Tuesday, February 1st, the Office of Information Security and Compliance will be rolling out two new initiatives designed to reduce the number of compromised hosts seen on campus. Those initiatives are:
Notifications of vulnerable Java installations
One of the more common exploits we see are attempts to compromise a system via flaws in old versions of the Java Runtime Environment (JRE).
These exploits often cause Java to attempt to download malicious files, and in doing so exposes the User-Agent string which we detect with our Intrusion Detection System (IDS):
================================================== ET CURRENT_EVENTS HTTP contacting a suspicious *.co.cc domain -------------------------------------------------------------- XX.XX.XX.XX | 3153 | 22.214.171.124 | 80 | tcp | 2011-01-29 15:42:53 -------------------------------------------------------------- GET /pk6/cralxzkvergm.jar HTTP/1.1 accept-encoding: pack200-gzip, gzip content-type: application/x-java-archive *** User-Agent: Mozilla/4.0 (Windows XP 5.1) Java/1.6.0_14 Host: phenomen02.co.cc Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Connection: keep-alive *** (the latest version is Java/1.6.0_23)
Starting on 2/1, when we detect this activity we will send out notices with instructions on where to download the latest version.
IDS driven TCP resets to tear down malicious connections
Our IDS has the ability to inject TCP resets into connections when a particular signature is detected, which essentially tears down and terminates the session. Unlike a blackhole route, this is done on a per-connection basis only.
Starting with the Java exploits, we will be sending out resets to both the source and destination hosts when an exploit attempt is detected in an attempt to prevent it from succeeding. Note that this will not affect connections where no exploit is detected.
Recently reset hosts and all blackholed hosts can be found here:
Please let us know if you have any questions at firstname.lastname@example.org.
Jim quickly backed off saying that the date this would begin had not yet been decided and he had "jumped the gun." Later he revised this saying it would start Tuesday, Feb. 8th.
In the meantime Wayne quickly published a new Power Tool to assist in locating vulnerable machines.
Jim Hranicky provided some additional feedback as well earlier this week:
Message to Net-Manager-L from Jim Hranicky
Sorry to keep talking to myself, but the resets are live:
curl -H 'Host: operasov.cz.cc' -H 'Connection: Keep-Alive' http://126.96.36.199/pdf3.php | strings % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (56) Failure when receiving data from the peer
The rules have been firing for a while, but the interface just went live.
Recent rules that have tripped are here:
If anyone else wants to try the curl command above and let me know what they see I'd be glad for the feedback.
One more thing and I'll stop, maybe.
If you are getting notifications on systems where you simply can't upgrade Java (sigh), let me know and I can add your host(s) or network(s) to an exceptions list.
Winnie asked if these notices should be treated as compromises. The consensus was that Java should be remove/updated as needed and perhaps upgrade to VSE 8.8 (the ePO console now has a tag for that) and run a full on-demand scan of the box. If the box was compromised it would be expected that additional alerts would likely be forthcoming.
Steve noted that there is a freely available JavaRa application which will remove all old and redundant versions of the Java Runtime Environment at once without you having to manually uninstall each separately via the Control Panel.
Steve also noted that SAS installs vulnerable versions of JRE and he has yet to figure out the workaround. Kevin Hill pointed out that SAS version 9.2 offers to use an existing version of JRE, but Steve said he has not yet been able to successfully use that. Kevin responded that he had done so without problem so Steve certainly needs to investigate further. Joel Parlin had mentioned trying to edit a sassw.cfg file to point to the newer version but later learned some aspects were still broken. Supposedly complete details are available but note that a release version specific path is involved making this a continuing hassle at each Java update. Steve is not familiar enough with SAS yet to get into the graphing portions where this JRE is used so he can test things.
In the meantime, Steve is simply removing the vulnerable JRE that SAS installs. He has found that the majority of users do not utilize SAS functions which require JRE--though there are exceptions.
Adobe also released Flash 10.2. Steve would like to reiterate that remote patching via PsExec seemed to work fairly well for Flash. His attempts to use a similar method with Java have been thwarted by the fact that the installation will fail if the user has IE open at the time. You can grab the various files here:
Steve added, however, that he has recently noticed that the uninstall utility does not fully remove Flash10l.ocx (current one is Flash10m.cx) as Live Messenger and/or Skype keep that file locked. Such issues with third-party patching continue to be an extreme ongoing hassle.
MS Office News update
Updates not available...
Job Matrix Update status
This is here as a standing topic--no discussion this month.
Updates not available...
Supporting wireless printers in a wallplate building?
Dennis Brown said that one of his users had asked about using a wireless printer and Dennis was looking into it. He had sent an email to Matt Grover and then submitted a ticket as well. It has been a few days and he is yet to hear back. Steve suggested that this should be discouraged because he believed such an implementation here would be more trouble than it was worth. Chris Leopold seemed to agree.
UAC settings egregious for users?
Updates not available...
PDF-Xchange (prior discussion)
Updates not available...
The meeting was adjourned on-time at about noon.