ICC Meeting:

IFAS COMPUTER COORDINATORS
(ICC)

NOTES FROM June 12th 2009 REGULAR MEETING

A meeting of the ICC was held on Friday, June 12th, 2009 in the ICS conference room. The meeting was chaired and called to order by Steve Lasley at about 10:00 am.

PRESENT: Seventeen members participated.

Remote participants: David Bauldree, Bill Black, Micah Bolen, Dan Christophy, Dan Cromer, Chris Fooshee, Louise Ryan, Mitch Thompson, and A. D. Walker.

On-site participants: Dennis Brown, Andrew Carey, Lance Cozart, Marion Douglas, Joe Hayden, Wayne Hyde, Steve Lasley, and James Moore.

STREAMING AUDIO: available here.

NOTES:

Agendas were distributed and the sign-up sheet was passed around.

Polycom woes at ICS

There was a problem with the Polycom at ICS and we had to drop out of the VC. Instead we phoned into the VC and used People + Content IP to distribute any computer output. From e-mailed comments, the audio may not have been very good via this makeshift method and for that we apologize. Perhaps the locally recorded audio is better if anyone wants to check things out that way.

Videoconferencing topic moved to top of our meetings

Dan Cromer had asked that we move our standing videoconference discussion up to the beginning of our meetings so that Patrick Pettus and Lance Cozart could more easily participate. That topic was indeed discussed early on this meeting but is linked into the notes at its previous location. That will change with the July meeting.

Report from the chairman

Member news:

New members

Jin Wang is replacing Raymond Cho as Systems Administrator for CALS under Wendy Williams.

Recap since last meeting:

As per his usual procedure, Steve pointed folks to the notes of the last meeting, without going into any details.

Policy

WebCT going away? (previous discussion)

Steve didn't know how things stood with this, but noted that Blackboard has been getting a lot of flak for snatching up Angel Learning. Perhaps it is good that we are going with Sakai, if Blackboard support has been so terrible.

WAN transition to CNS (previous discussion)

Clarification of critical vs. non-critical monitoring

Most IFAS sites operate under what CNS terms "non-critical monitoring", though any site may request "critical" monitoring. In the latter case, CNS on-call staff will be notified and will phone local support whenever their automatic monitoring system notes an outage. Critical monitoring generally only makes sense if the local support person is willing and able to visit the problem location 24/7 for diagnosis.

Non-critical sites still receive rapid response during the work day, and after hours alerts are queued until the morning of the next business day. Regarding any outage, CNS won't call the telco until they can be sure power is on, since that's the most common cause of an outage.

If some site has special needs regarding notification, CNS is willing to work with you to meet those needs.

Power outage monitoring

Regarding any critical monitoring sites that may exist, Joe Hayden recommended that FPO be contacted first because they will know when power goes down. They have sufficient UPS at each site for them to get data on power outages. When Steve asked, Joe clarified that these were at major remote units, not CEOs.

Regarding the CEOs, James Moore said that one of their goals is to replace all the routers at the CEO office as soon as possible. Those replacement routers have what is known as a "Dying Gasp" feature. When the router is about to lose power, this function detects the situation and sends a power failure indication signal. A UPS could still be the problem (vs. true power outage at the site) but at least they should have some better information upon which to act.

Network equipment census status

Steve asked if any of the RECs had begun documenting their existing commodity equipment--what James terms "transparent devices" which CNS cannot see with their monitoring equipment. Doing that is critical to evaluating the costs and consequences of adopting the proposed SLO. A. D. Walker was the only REC support person represented at the meeting and he seemed unaware of the remediation requirements that the SLO might entail. Regarding the CEOs, James said that the district support folks had pretty good contacts at each site and should be able to obtain pictures from them to help document things there. He also can turn on LLDP to help map out things at least to the managed switch and in some cases somewhat beyond (where edge devices support spanning tree).

Steve mentioned that he had been expecting Dan Miller to put out another plea for getting things documented. James responded that Dan has been inundated with work related to writing grant proposals and end-of-year money expenditures. We can rest assured that they have not forgotten about the issue, however.

Some discussion ensued about how best to coordinate the distribution and updating of our network equipment inventory. Joe Hayden particularly asked that FPO be involved as they make fairly frequent site visits and could verify CNS's understanding of things and help keep that updated. James said that he would request that FPO be given access to the wiki so they could see the network diagrams, network topologies and the IP space utilization. Joe said that this would help both FPO and CNS with troubleshooting.

IFAS WAN status updates

James reported that six sites previously supported by FIRN will have AT&T and MFN as their network providers. James has put in the request to DOE to disconnect those FIRN circuits. Four of the sites will get T1's under two-year agreements: Hastings, Marianna, Jay and Brooksville.

At Marianna, A. D. Walker has worked out an arrangement with Star2Star for an ip phone setup; this caught CNS a bit by surprise, but CNS is trying to help them out in spending end-of-year money. VoIP there will use about 1/3 of their T1 and they are going to allow six lines to be up simultaneously with the capability of a burst up to possibly ten; doing that will require 350Kbps or greater. James is not aware of what sort of compression will be used at the head-end, but he is being told the data will amount to 32K call-to-call. Marianna's existing T1 is already flat topped and because a T1 is a shared line there is not much that can be done regarding QoS. James plans to set a default route one way and use policy routing in the other direction. Marianna's T1 has a 1.5 symmetrical CIR. James is going to take measures to prevent Polycom usage from impinging on their VoIP traffic which will have a priority queue.

All the data back to campus and their general Internet traffic will go over their 6 Mbps DSL (1.5Mbps upstream). There may be something down the road that James can do to allow true split-routing/load balancing between the T1 and DSL networks; that may lead to a second router deployment there but he will just have to see.

The Brevard CEO in Cocoa and the Madison CEO will be getting cable connections via Brighthouse. Brighthouse has a new program that offers to run fiber in at their own expense to try and draw new business in Florida; that would provide even more and better options. It was very close with Cocoa, but Brighthouse wanted a $900/mo 3-year service commitment for a fiber pull that was just outside of the CEO's $500 budget range. Instead, they will be getting a 15x2 (15Mbps down 2 Mbps up) cable connection.

AT&T already had fiber at the Plant Science Unit at Citra and was willing to pay for the conduit to pull it into the building. They will be getting a 10Mbps burstable to 100Mbps "metro" connection. CNS was able to haggle AT&T down to $525/month for that service.

James mentioned that Homestead is another location with an AT&T 10Mbps connection; though it had taken a while to get that all arranged. Ft. Lauderdale, Ft. Pierce and Homestead all have these 10/100 burstable connections back to a 100Mbps FLR backbone (actually 1 GB with an 100 Mbps interface). CNS carves out a 10Mbs VLAN for each of these sites. This arrangement allows other UF groups to help share the costs by placing them on the same 100Mbps circuit over FLR as well. Being able to pool such things will provide leverage in negotiating prices with the "last mile" providers down the road.

CNS is also looking at various grant opportunities to assist with paying for our network infrastructure across the state and is working with Pete Vergot and Dan Cromer on a couple of possibilities for IFAS currently. CNS also continues to monitor other service providers to try and get us the best deals possible. T1's and the best DSL/cable combination will be a stop-gap until we can get fiber pulls to all sites. Milton is going to get a fiber connection soon, for example; at other places like Jay, however, the best they can get due to distance from the DSLAM is DSL with 3Mbps down and 386Kbps up.

Joe Hayden alerted James to a pressing issue at GCREC's Plant City campus. The community college that hosts us there is going VoIP on very short notice and we need to figure out how our side of that can work. At this point it appears we need to stick with our copper system, but that will lead to coordination issues with the rest of campus.

James said that CNS has put a bit of money aside for IFAS WAN equipment remediation and Dan Cromer threw $10k into the pot as well. They are beginning with WAPs and are looking at money for cabling and wiring. There might be some funds available, so James encouraged support staff to assess locations where Cat5 drops might be needed to eliminate hubs and get that information to him.

Renumbering Ft. Pierce

James explained that the renumber which was occurring at Ft. Pierce simultaneously with our meeting was due to them requiring additional address space and having been allotted that. Joe Hayden expressed concern that such things be coordinated with FPO because they have equipment at many locations with statically assigned numbers. James said that the mid-day changeover was a bit unusual as they generally like to do such moves outside of normal work hours; Bob Huston made the request, however, and CNS tries to accommodate. Joe noted that local IT staff may not realize the full networking picture. Ft. Pierce has a phone system which is statically addressed, for example. Joe and Chris Leopold arranged dual numbering temporarily until they can get the phone vendor in there to fix it, but this is just an example of the coordination needed.

Joe gave Citra as another example. FPO has a conduit which goes all the way out to the road there because of a new administration building which will eventually be built there. FPO has stuff in the ground that is not well documented and if CNS brings a new provider in that creates a new trench and FPO is not aware, the chances of it getting cut down the road are fairly high.

Six new MPS sites by August

Andrew Carey asked James if he was aware of these new MPS CEO locations: Leon, Okaloosa, Santa Rosa, Nassau, Flagler, and Hillsborough which currently don't have routers. James responded that Chris Leopold had informed him of this and he is working with Louise Ryan to get a router at Okaloosa. They currently have a Linksys WAP, but CNS will be deploying a lightweight Cisco WAP there which will authenticate back to UF. Unfortunately, that configuration will send all traffic back to campus too, which isn't great for performance. In the interim James intends to support the Linksys WAP and see if he can configure that as pass through.

New myuf Market requisitioning system changeover beginning July 1st (previous discussion)

Steve asked for input on this topic, wondering how other units felt about this. His own unit's fiscal group is quite concerned at the extra work they feel this will entail for them. Joe Hayden said that his group was hesitant initially, but they have been using it for eight weeks now and have grown to love it. Dennis Brown mentioned that he had attended the training but did not get much out of it.

Dan Cromer mentioned that he attended the training and that he feels it will making buying easier. He is concerned, however, that they system may not lead users to getting the best deal, rather they will go for the vender whose presence in the system makes purchasing easier. We'll all just have to see how things go as more units adopt the system for use.

UF IT Action Plan (previous discussion)

Steve noted that he had no news beyond Dr. Frazier's "Open Letter to UF IT Staff" in the June edition of IT Connections. The ITAC-NI is one committee that is no longer meeting while the new committee structure gets worked out.

UF Exchange Project updates (previous discussion)

Office Communications Server coming to UF

Dan Cromer announced that this will be going into production mode in July. Patrick Pettus with John Pankow has been working to make the Polycom endpoints accessible to the Office Communication Services (OCS). Tandberg equipment can do that, but Polycom has issues.

Utilizing OCS requires that a user's account in UFAD be defined as SIP qualified; that in-turn requires a $9 years license for the enterprise CAL. Mike Conlon did not want the enabling of that for individuals to become an administrative headache for UFAD support staff; consequently, he has mandated that this be implemented only at the unit (OU?) level within which all or none participate.

Steve suggested that Dan might lobby for OCS licenses for those in IFAS with IF-ADMN accounts so this might be available as least for IFAS IT support. That might be a minimal cost which Dr. Joyce would be willing to support if we could get Mike Conlon to waive the literal "per department" requirement just a bit.

Dan mentioned concerns over the bandwidth which OCS might grab over our WAN links as well. The way the client handles that is apparently not bandwidth tunable and it will grab whatever bandwidth appears to be available in order to provide the client with the best connection.

Steve said that he is not sure what departments would be willing to foot licenses for their entire unit based on just OCS. The enterprise CALs provide full client access to all MS server products, however, including MOSS; the combination of potential capabilities might provide greater incentive beyond just this single piece. For example, Steve has some interest in Microsoft Application Virtualization, former known as SoftGrid. Enterprise CALs would open up a host of opportunities.

Split DNS solution for UFAD problems

Steve wants to keep this on the agenda for future reference.

Projects

IFAS WebDAV implementation

There continues to be no progress on the documentation which was to happen prior to announcement. Since this has never been formally announced, the matter remains on the agenda as a standing item.

Vista Deployment via SMS and WDS

Steve noted that he had changed this standing agenda item from "Vista Deployment" to "Windows 7 Deployment. He wanted to get Andrew's view on this because he knows ITSA does not have the time to be building desktop images for the various units. Rather, Steve asked if Andrew might have time to provide WDS support on the server side if we could arrange some ICC volunteers to spearhead an effort.

There are new tools for deploying Windows 7 with some very powerful features which all IFAS could benefit from if we could get a small group of people willing to dedicate some time to learning and implementing them. Andrew said that he has been working with the new Deployment Toolkit 2010 Beta 1 in relation to the new MPS deployment. Windows 7 and Windows Server 2008 SP2 both use the same basic kernel and these new tools apply equally to both platforms.

Andrew believes that it would be very useful if some of the ICC began looking at this as well and he would lend assistance as his time permitted. Steve noted that Micah Bolen had volunteered and that Lance Cozart had prior interest which was side tracked by other duties. Steve hopes Micah or someone else can take the lead in organizing that and in getting together interested folks to talk over what might be done.

Steve noted that Windows 7 brings some changes regarding deployment. In particular, one can no longer manually copy profiles over the default; that function is grayed-out in the user interface. Apparently, this has not been recommended practice for some time, but Steve has always used that method to control the users' configuration at first logon.

A PXE boot deployment of a basic Windows 7 installation would be a huge timesaver for everyone, including the Help Desk that has to support so many different units. When Lance brought up application deployment, Andrew mentioned that we may be able to make use of System Configuration Manager (former SMS) for that aspect. He believes that our campus license may cover the CALs for that and if so all we would need is the server.

Exit processes, NMB and permission removal (prior discussion)

Nothing further was available on this topic at this time.

Re-enabling the Windows firewall

This is still planned but is pending the time to implement.

Operations

Wayne's Power Tools (previous discussion)

Wayne Hyde continues to improve tools which will be a great help with managing OUs. Steve wanted to encourage those who have not tried them out to do so.

The "Track User logins" and "Track Computer logins" tools use Lansweeper queries to allow you to see who is using what machines--you no longer have to explain to the user how to find what their machine name is because you can run a query which will locate them and tell you when they logged on where. You can even find those machines which aren't logging Lansweeper data by using the "Local Group Members" tool and looking at the bottom of the output provided. That gives you a foot up on resolving any such issues for your own OU.

The "Computer info" tool gives you access to the complete hardware and software inventory for all your computers. That includes all the autoruns, services and processes which were running at the time the computer was last scanned. That information can greatly assist with diagnostics. Wayne plans to have the items on the software list clickable so they will return a list of all machines in your OU which match that item.

Chris Leopold developed a web-version of his DHCP search tool in C# under .Net. As a result, Wayne is working to move his tools from classic ASP to .Net 3.5 so both sets of tools can be supported in a single web interface. The long term goal is to develop self-help tools which will allow OU Admins to create things like printer shares via a self-service web site. That will save time for everyone in the long run. This will involve some fairly tricky programming and a lot of up-front learning to pull off, however, so don't expect quick progress.

Folder permissioning on the IFAS file server

You are reminded to please take the time to read and implement the new standards. If you have any questions get with Wayne or Steve.

Disabling/deleting computer accounts based on computer password age

As with so many things in these times of inadequate staffing, finding time for implementation is proving difficult.

New MPS/DC testing -- access by unit-level administrators

The six new MPS sites mentioned earlier as well as the other existing MPS locations will be virtualized along with a DC in one box per location. Seventy-four have been ordered and received. They had asked district support to locate sites that did not currently have MPS/DCs and which wanted them; Andrew was happy to report that they were able to supply all sites wanting those. ITSA is waiting for Windows Server 2009 R2 to RTM before they deploy; between now and then they are basically tuning their performance and doing test deployments potentially at several locations in district two due to its proximity to campus.

Andrew added that all the sites will be getting a new APC Smart-UPS 1500s as part of this deployment; those should give us a bit more up-time over the less capable units which they are replacing. One problem is that they weight roughly 60lbs each; district support may have to rent trucks to handle the load for deployment. Joe Hayden suggested that ITSA might want to send those out with his guys as they visit most center locations across the state with a one-ton truck on a fairly frequent basis, so load is not an issue. That would at least get them as far as the various centers so that district support might have an easier time of things.

Report generating system

Steve mentioned that Wayne's Power Tools are addressing many of the concerns here and that this item might be taken off future agendas as a consequence.

Core Services status

Wayne mentioned that he has some updates to do on the ESX servers to get them on the latest versions--including VMware View. These are not time sensitive updates but he wants to get them done. Other than that things are in steady maintenance mode.

ePO version 4 status

This topic was not addressed this month.

Status of SharePoint services (prior discussion)

Steve did note that this is being developed at the UF level now; he had hoped for an update on that and on the status of support for external authentication, but Ben Beach was not available.

Public folder file deletion policies and procedures status

Nothing further was available on this topic at this time.

Videoconferencing topics

Comments resulting from yesterday's video meeting

Dan Cromer reported that a productive meeting had been held yesterday which included Marion Douglas and Lance Cozart. John Pankow had provided a document which Dan Cromer will be sending to the ICC; that talks about the division of support responsibilities between IFAS and VCS. Lance will be handling some of the things on campus, including ordering for and configuring campus rooms.

Polycom inventory needed

Dan mentioned that job one is to get our inventory of Polycom equipment in order; he may hire OPS to assist with that aspect. If Dr. Joyce approves, Dan hopes to send out a message to all the unit heads alerting them to our need for inventorying this equipment and asking that they identify an individual as the official video conferencing contact at each site. Those will be the people we hope can be contacted to provide the needed inventory details--particularly to get the equipment serial numbers recorded centrally.

Steve asked whether District Support would be bypassed on this. Dan responded that he wanted them involved, but did not want them to have to do the phone calls to get this serial number info. Dan feels identifying a local contact would be more efficient.

Dan is also interested in obtaining photos of all the CEO Polycom equipment at the CEOs. Ben Beach has already done this for his district and has posted it on SharePoint. The equipment and its configuration varies from office to office and it can be a great help for support to have a good idea of what is located where and how it is all connected.

James Moore expressed interest in having pictures of the networking equipment taken during this same effort. Steve agreed that it would be most efficient to address both the Polycom and network equipment aspects similarly as there are needs for a complete inventory in both cases.

James mentioned that CNS will be generating network topology maps and making them available via a wiki. He would like to give VCS access to that so they can provide feedback if they see something in those which is not accurate.

Polycom configurations now locked

Steve asked whether it is true that local support will be locked out of the Polycom system settings. Lance responded that Patrick will need to be contacted whenever any configuration changes are needed as those passwords have been changed.

Polycoms to be left on and connected 24/7

Besides configuration access, the other major change is a policy for leaving units on and connected at all times for maintenance purposes. Mitch Thompson, however, felt that mobile units should be able to be stored away in a closet--that this was pretty much the point of them. Lance said he is interested in getting the mobile Polycom units mounted in a fixed location because moving those leads to equipment failures. If systems are to be stored in closets, however, provision needs to be made for power and network connection so they can remain on. Joe Hayden pointed out that ventilation will be an issue as well so equipment doesn't overheat.

Maintenance window for updates

Louise Ryan noted that one of her users was bumped from a VC due to an unannounced software update which was being pushed out. Louise asked that these be coordinated and Steve recommended developing maintenance windows when such things might be expected and could be handled without conflict.

Lance said that trying to get automatic updates arranged via TMS has been challenging. One aspect is that TMS doesn't interface well with our Polycom endpoints. Another is that we don't have the correct serial numbers and keycodes in our database to permit that. Finally, a number of units are either turned off or behind firewalls and not reachable. Apparently, however, Patrick has turned automatic updates on for at least some units recently. Coordination is still needed to make sure that doesn't interfere with endpoint usage, however.

Equipment decal issues

Apparently the decal numbers and serial numbers for many of our Polycom units are in somewhat of a mess. Some have multiple decals (likely due to other associated equipment purchased at the same time whose decals ended up on the Polycom unit itself), others have no decals or the Asset Management database has essentially made-up information useless for identification purposes (Lance estimated this at about 50%!). Joe Hayden commented that we really need to do annual inventories of decaled items. His group is involved in trying to do that throughout the state and it is a real challenge. Equipment is often missed in the first round of scanning, which requires sending someone out a second time. Joe mentioned that if Polycom equipment is being swapped out, the serial numbers and decals involved need to be coordinated with Michele Barr at FPO so they can keep track of things.

Handling of recorded VC files

Dennis Brown had requested that IFAS create a means to off-load recordings from the VCS server to IFAS for long term storage and access. Dan Cromer had mentioned the possibility of setting up a server supporting NFS for that purpose. Wayne is concerned about how much space these might take, but also about having to support such a service on hardware that is off-maintenance.

Steve proposed that steps be taken to assure that these files didn't just accumulate without periodic checks on whether or not they still needed to be maintained on-line. He suspects that long term access is not needed in most cases and that each department could maintain an off-line archive from which items could be restored from should access be needed for a certain period.

Comments from Dean Delker regarding People + Content IP

Dean had provided Steve the following comments via e-mail which others may want to keep in mind:

Some people use the term “People + Content” generically, meaning talking heads on the normal video channel and PC output on a second higher def channel. That is best done with hardware like a Video Concert with a VSX 7000 or built-in with the 8000’s.

“People + Content IP” is a proprietary Polycom software product that sends the PC video output through your LAN to the Polycom’s IP address. I’ve seen people have trouble with that. Especially if they already have bandwidth challenges. It seems to take a lot more overhead than the hardware implementations.

Also Patrick & I have seen some problems with People + Content IP restarting if the Polycom endpoint drops out and adds themselves back in. It doesn’t hang up properly on the bridge, so it won’t restart easily. I’m not even sure of the best way to reinstate it.

Those of you relying on People + Content IP may want to keep this in mind; should your endpoint drop out of a bridged VC you may have an issue getting your content to go upon reconnection.

Patching updates...

Microsoft

June was a heavy month for Microsoft patches with six critical and three important updates for Windows, IE, and Office. You might enjoy the podcast "Security Bulletins for the regular IT guy" produced by some Canadian security folks; it provides a nice summary of these patches.

Apple

A QuickTime security update pushes the latest version to 7.6.2.

Adobe

Adobe recommends updating Acrobat to versions 7.1.3, 8.1.6, or 9.1.2. Those using Adobe Reader should replace older versions, moving to 9.1.2.

Java

Java Release 6 Update 14 was just released. Apparently this was not a security-driven release but does contain main bug fixes and performance enhancements.

MS Office News update

Steve said that there have been a few reports of issues with Office 2007 SP2 (breaking desktop alerts in Outlook 2007 for one) though he hasn't seen any problems personally.

Job Matrix Update status

There was no time to address this topic.

Remedy system status

Steve wants to leave this matter as a standing agenda item for future discussion.