ICC Meeting:

IFAS COMPUTER COORDINATORS
(ICC)

NOTES FROM August 10th 2007       REGULAR MEETING

A meeting of the ICC was held on Friday, August 10th, 2007. The meeting was chaired and called to order by Steve Lasley, roughly 10:15 a.m. in the ICS conference room.

PRESENT: Twenty members participated--the majority remotely.
Remote participants: Julio Bastidas, Bill Black, Allan Burrage, Trish Capps, Andrew Carey, Rick Faulk, Chris Fooshee, Kevin Hill, Wayne Hyde, Dwight Jesseman, Chris Leopold, Mike Ryabin, Mitch Thompson, and Paul Weikel.
On-site participants: David Bauldree, Dennis Brown, Dan Cromer, Winnie Lante, Steve Lasley, Wendy Williams.

Special guest: Doug Johnson

STREAMING AUDIO: available here

NOTES:

Agendas were distributed, and the meeting began a bit late. We had problems with laptop projection locally as well as with the audio streaming. Though unable to stream, audio was recorded locally, however, with many thanks to the efforts of Dan Cromer; consequently that recording is now available. My apologies for those unanticipated difficulties; unfortunately, problems of one kind or another seem to be the norm.

Prior to the meeting Steve meant to share one of his favorite Strong Bad toons but the technical difficulties made that impossible. Many will feel themselves fortunate, but those who do not please let Steve know. He is always looking for ways to add interest to our meetings.

Report from the chairman

New members:

Steve noted that Stewart Collins is the new IT support person for Microbiology & Cell Science. Daryl Grimm used to provide support both there and at Agronomy. When Daryl took a programming position in Boca Raton, Agronomy hired Gregg Knapp. Now Micro has apparently followed suit with Stewart. So our ranks are growing!

Also, as mentioned briefly by Dan Cromer at our last meeting, William (Rick) Lusher has replaced Larry Treadaway as manager of FAWN.

We'd like to welcome both Stewart and Rick. We hope they choose to participate with us in the future.

ICC Alumni News:

John Sawyer, former IFAS security engineer prior to Wayne Hyde, and currently with UF's security team was on the winning team again in the DefCon "Capture the Flag" contest. This makes the second year in a row! John has also been busy writing for Information Week and has started an "Evil Bits" blog at DarkReading.com.

If any of you other alums ever have any news to share, just let Steve know. We would all like to keep in touch with you.

Recap since last meeting:

As per his usual procedure, Steve pointed folks to the notes of the last meeting, without going into any details.

Special Guest

Why doesn't IFAS use WebCT?

At our June meeting, some discussion arose over the question "Why doesn't IFAS use WebCT?". Of course, that isn't universally true, but there did seem to be a prevalent impression that WebCT was too difficult to use and that it had performance issues which interfered with its instructional use--at those very times when the instructors were most depending on it. From that discussion, Steve offered to invite Douglas Johnson to come speak with us. Doug is Manager of eLearning Support Services at UF's Academic Technologies, and he kindly took time from his busy schedule to visit with us here at this month's meeting. Doug has been supporting web-based instruction here at UF for the past seven years.

Doug and his staff have been frustrated lately by the slow progress on major efforts to move from the old version of Vista (Vista 3) to the new Blackboard Vista 4 (to be dubbed "eLearning"). (Note: a web-accessible video is available which explains the feature changes to expect between the two versions.) In fact, the database conversion for that completed successfully only this very morning at 4am. You may read about some of those travails in Doug's Vista Transition Blog.

Doug began by admitting that he really has no way of specifically answering the question "Why doesn't IFAS use WebCT?". The true answer to that lies with the end users. He did note, however, that one major factor may indeed be the fact that WebCT encompasses a rather large set of tools, and as with any software of such complexity there is a learning curve to its use. Many instructors who are already familiar with building web-based materials and uploading those locally find difficulty--if not just plain lack of interest--in learning a new system for doing that.

Overview of tools offered

Doug explained that WebCT/Vista is comprised of roughly 25 different tools combined within a single web-based interface for delivering instruction over the Internet (all or in part). While there is a wide range of use, Doug believes that the majority use Vista for nothing more than as a means for posting their syllabus and grades. At the same time, there are fully robust complete on-line courses offered via Vista, including degree programs at both undergrad and graduate levels. How a particular instructor utilizes the system is really up to them; eLearning Support Services is there to support whatever level instructors may bring to that system.

The tools provided cover a wide range of needs. At the simplest level, they provide a simple file-to-web uploading interface whereby an instructor may utilize the web to distribute documents pertinent to their course. A more sophisticated tool, called the "Learning Module", is available that provides an easily managed navigational structure for presenting materials on-line in whatever flow-structure an instructor may desire. This obviates the need for an instructor to understand the HTML programming aspects of creating a particular navigational system; rather they may simply upload the content in a desired order and let the system manage the navigational programming while providing a flexible interface for controlling/changing that.

Another advantage to these Learning Modules is that they allow instructors to track information about course usage. By viewing the tracking information an instructor may see which links students are and are not viewing and how much time is spent in each (although that feature shows only the time spent with a page open--not whether or not the viewer is actually studying the content there). This would allow an instructor to track, for example, whether or not a particular student took advantage of the materials being offered. This has proved valuable in the past for instructors faced with student complaints that a particular test topic was "unfair" due to it not having been covered in the course. This tracking data can also be a useful tool for instructors in determining whether or not the content being offered is pertinent to the learning objectives. If students are doing well on a particular topic despite many not viewing that content on-line, then this may indicate that those materials are either not very useful and may be removed, or that they can change their assessment of the topic to make it clear to the students that this information is something which they should be looking at. Thus, tracking information provides a means for instructors to improve self-assessment as to how effective their methods are in reaching their students.

Another tool within Vista is the chat tool, so you may have synchronous chat. Virtual office hours are supported as an adjunct to chat usage. Discussion boards are also available for asynchronous threaded discussion. Many instructors use the discussion tools extensively, because it allows them to inform students about necessary preparations for upcoming classes--materials to read, questions to answer, and the like. Students can use that to post their answers to questions which an instructor may review prior to class. This allows an instructor to better gauge student understanding and to thus tailor in-class instruction to their particular learning issues.

On-line quizzing is another feature supported by Vista. As an adjunct to that, Respondus software supports the off-line development of test materials which may then be uploaded into the system to provide on-line testing. Some courses use frequent quizzes to assess student understanding in much the same way as mentioned previously regarding discussion boards. Again, this provides instructors assessment tools which may be utilized to better tailor in-class instruction to particular student needs.

Steve asked if the Vista system provided support for tying learning objectives to quiz content. Doug responded that this has not been available with their current system, but that it is being introduced with Vista 4. The idea here is that courses be built around specific written objectives. By tying quiz questions to those objectives, both students and instructors can assess mastery of those via a series of pre- and post- quizzes. This has been a feature of computer-assisted instructional management since its earliest days in the mid-to-late 1970's.

Thus, Vista provides a broad set of tools which an instructor may utilize to assist in the instructional process. How those are actually being utilized varies widely.

The taxonomy of web-based instruction at UF

A Distance Education & eLearning Task Force was recently appointed in order to recommend a direction and strategy for distance education and eLearning at the University of Florida. Part of their task has been to develop a common set of terminologies to describe a taxonomy of the various types of programs which might be suitable to our needs. There are over 5000 courses currently within Vista, managed by over 3000 instructors. About 80% of those fit into the category "web enhanced course". This category includes courses which are fully face-to-face courses that utilize on-line elements such as syllabus and document posting (e.g., PowerPoint lectures) to support traditional methods.

A "hybrid course" utilizes the web to replace some in-class activities with on-line activities. This allows instructors to decrease the amount of "seat time" and hopefully increase the amount of student engagement and activity with the course content itself. Roughly 4-5% of our current Vista-supported courses fit into this category.

The third category of course is termed a "web-based course" or "distance education". These courses make up about 15% of our current Vista courses and include courses where practically all instruction and interaction takes place in the on-line environment.

About 40% of the courses taught at UF have a presence within the Vista course management system, or roughly 2900-3000 active sections for a given Spring or Fall term. There are a number of initiatives around campus which Doug feels may push those numbers higher. The Provost's office has just begun a program to address a number of the chronically over-subscribed course offerings--those required courses which may hold up the degree progress of numerous students if demands cannot be met. This program intends to create fully on-line versions of these courses and thus give students the option of taking those via that method rather than via conventional in-class instruction.

Doug mentioned that his experience shows that students quickly gravitate to the on-line options when they are offered. He has seen this with the College of Business and their online course offerings. Though all students have a physical seat in a classroom, all courses are also available on-line and attendance in the physical classroom is optional. After the first test, typically 95% of students no longer attend the physical classes. Statistics is another department that has offered on-line options to traditional classes, with the finding that a majority of students rapidly chose the on-line experience over the classroom experience. In fact, they have shown that on-line students (on average) do better in the course than do those attending traditional in-class instruction. The hypothesis there is that on-line students take the opportunity to review the material multiple times and therefore assimilate the information better. As result, he expects considerable growth in web-based instruction at UF.

Dan Cromer asked what is done to prevent students from collaborating in the taking of on-line courses. Doug said that, in the Intro to Statistics class, testing is still done via the traditional in-class method whereby ID must be presented to reasonably assure that the person taking the test is the person enrolled. The other aspect to this is that many instructors do not care if a student "cheats" to learn the content--as long as they do indeed learn it.

The Vista hardware platform

The Vista 3 system runs on RedHat Enterprise Linux AS3 with the hardware being divided into three tiers:

• Data Tier

This tier runs on a single IBM x445 with 8 processors and 32GB of RAM pointing to a storage array which has 4TB of storage in a mirrored arrangement providing 2TB of usable space. 1.4TB of that space has been consumed by data within the Vista system.

• Application Tier

This tier is comprised of 13 IBM x345 servers, each of which has 4 processors and 2GB of RAM. They have about 73GB of onboard storage, but that is used for nothing more than running the applications on those servers.

• Administrative Tier

The final tier is a WebLogic node that manages the J2EE environment in which this entire system operates.

The system uses software load balancing provided by Network Dispatcher. Two years ago Blackboard bought out WebCT in a move that would be analogous to a situation where Coke might buyout Pepsi. This has caused a great deal of grief in a number of different contexts--with load balancing being one of those. Doug pointed out that the Health Science Center had previously managed their own Blackboard service. Recent cost increases has led to them moving over to Vista on a voluntary unit-by-unit basis.

Migration to the new Vista 4

One of the most exciting components of this upgrade for Doug is the opportunity to replace a single data store with a more robust and redundant solution. Doug has been worried for the past four years about having the possibility of a single point of failure for the entire system. From the beginning of this project in the summer of 2003 they had proposed clustering the data tier for fail-over in the case of a major hardware failure. It was only in March of 2007 that they finally received the funding to be able to do that.

The upgrade project will involve all new hardware for both the data and application tiers. There will be four clustered data stores on the backend and the data storage will be moved from the current IBM EXP400 storage array to a clustered SAN solution; there will be two separate mirrored SANs in this scheme. This will provide the increased speed and growth opportunities of a SAN in addition to the data redundancy such clustering will provide. LSS is moving from independent stand-alone servers to HP blade systems. They will also be deploying VMware for the applications servers to allow easier consolidation and management of the multiple servers across various physical systems. This system will provide the capability of scaling to 24 application servers.

The multi-tiered backup strategy of the current system has one component that is highly processor intensive. Consequently, every time they run a backup everything slows down. Even though they have tried to schedule such backups at low-load times, finding such intervals has been increasingly difficult with an increasingly 24/7 pattern of client usage literally world-wide. Currently server level backups are run using Oracle 9i's RMAN. The upgrade will move us to Oracle 10g which will provide some advantages and efficiencies. Currently the system is run on RH Linux AS3 and they will be moving to AS4 64-bit (data tier) and 32-bit (application tier). This has been a long battle of contention with Blackboard to get them to support that newer platform and this upgrade will realize this goal. Significant improvements in overall performance should result from all these changes.

Conversion Problems

While Blackboard recommends a year-long upgrade process, UF had originally been trying to condense this into a 4 month March-June period. Through great effort, all the hardware and networking aspects were in place so that all that was needed was a database conversion process. Unfortunately, that process failed miserably. This caused a change of plans from a Summer B deployment where we would have had time to learn Vista 4 with a much lighter load level, to a Fall deployment.

They have spent a great deal of time since that initial failure in working with Blackboard and were finally successful at about 4am this very morning. They have been running into multiple problems with the upgrade script. Though the script had been well tested by both Blackboard and by UF, it had never been run on a dataset as large or apparently as problematic as our 1.4TB of stuff. Most of last week was spent in running the script, hitting an error, fixing the problem, and going on. There were single files which were corrupted in some fashion which would cause the script to fail and multiple incidences of these had to be laboriously eliminated. We now have a complete conversion.

The go/no-go decision

Consequently, they decided to immediately make a complete backup of the conversion so they will not have to go through that again. That backup typically takes between eight and ten hours. As soon as that is done, they will release the system for evaluation by Doug's staff (for features and functions) and by the Vista Institutional Professionals (VIP) group (for content). This VIP group consists of mostly college-level professionals; in the case of IFAS this includes Ron Thomas and Aaron Sotala from IFAS Communication Services. If things go according to plan, Doug will elicit feedback from these administrators who will give Doug a go/no go decision. In the case of a "no-go" decision they will drop back to the Vista 3 system for another term. If they get a "go" decision, they will take Vista down, upgrade the database from Oracle 9 to 10 and will also copy the data from the private IP based system which was used during conversion for security reasons, to its final production location. The hope is to open Vista 4, re-branded as "eLearning", by late Tuesday. Since they were up against a name change anyway from WebCT to the Blackboard Enterprise System Vista Learning Edition, they decided that local re-branding at this time made good sense. The goal then, is to announce the new system as open for business by early Wednesday morning.

New authentication system

Doug had mentioned that this was to be a comprehensive upgrade and he wasn't kidding. The Gatorlink authentication system is currently somewhere between 5 and 7 years old. The code that provides that service was written specifically to take advantage of that specific hardware. While there have been moves to upgrade the hardware, there has been no similar move to upgrade the software due to the fact that UF has been proposing to move to a centralized authentication system based on Active Directory. Unfortunately we have a ways to go with that process. What is happening right now is that when Vista "hiccups" we have large numbers of users who try to resolve that by logging back in. When up to 7000 users try to do that simultaneously it is overwhelming the Gatorlink Authentication server. That results not only in problems with people logging into Vista, but affects all Gatorlink authentication across its many uses here at UF. Consequently, most of the issues with the Gatorlink authentication over the past year have Vista ultimately as the root cause. Tim Fitzpatrick recently estimated that one-third of all GL Auth traffic is due to Vista by itself.

Because of this matter, they have decided to implement what they consider to be a short-term solution that will help things until the centralized authentication system gets revamped. The solution involves hardware and software to run CoSign, an open source project originally designed to provide the University of Michigan with a secure single sign-on web authentication system. This is a highly scalable system which can be grown to meet whatever demands Vista may place on it. What they are doing currently is modifying the CoSign code so that it will generate both Gatorlink cookies and CoSign specific cookies necessary for the CoSign software to function. This will allow users to move from Vista to other Gatorlink authenticated systems within the single sign-on model. To users, this CoSign-related change should remain transparent.

Users will see a different logon procedure, however, due to anti-phishing/branding considerations at UF. The current embedded logon at the eLearning site will be replaced by a link to a standard URL with the new UF "look and feel". This latter change will occur next week.

Obtaining access to Vista

Finally, Doug offered each of the ICCers access to the Vista system, should they not already have that. All that is required is to contact Doug via e-mail and provide him your Gatorlink username and/or UFID number. He will see to it that you are granted access to the system so that you may investigate it on your own. Let him know if you have a particular course you wish to move into Vista, if you have a course in mind which you wish to generate in Vista, or if you just want generic account in order to play with Vista.

Doug also mentioned that he is always happy to address any questions which you may have concerning the Vista system. With regards to on-site training, his staff is also willing to provide that for small groups of individuals both on campus and remotely. Those at remote centers would only be asked to provide for the cost of overnight accommodations--the training itself would be free of charge.

Feedback and Questions from IFAS Support Staff

Support for non-traditional education

Kevin Hill at SWFREC in Immokalee asked about what is being done to address the needs of extension non-traditional training. IFAS has numerous training needs that involve an audience of people who do not have Gatorlink credentials and Vista access is currently primarily based on such accounts.

Doug mentioned that a number of things are being investigated in this regard. One of the major initiatives involves changing the Gatorlink account process to include Level of Assurance attributes in the UF Directory that would allow accounts where strong identity is not required. This holds potential for allowing us to control access from our general public clients. Doug related that, while we do have this agreement in principle, there is no telling how long it will take to actually implement this in some fashion useful to our needs.

In the short run they are looking at two possibilities, one of which they can implement immediately. For an extension program that requires a discrete number of generic IDs, those can be created for you and can be associated with a particular course. Then that program can allocate those IDs as they see fit. Doug cautioned that the administration of the IDs (password creation, resets, etc.) would have to be the responsibility of the program--as the eLearning group does not have sufficient staff to handle that for folks. This is already being done, for example for FSHN department with their GNC University program.

The other thing being investigated, and for which they are seeking funding, is to launch a second course management system. This might be another instance of Vista or possibly an open source offering such as Moodle or Sakai and to use that system for programs whose needs simply cannot be met (for whatever reason) by the current Vista system. That might include a particular level of customization that would require separate handling. Obviously this requires a level of funding for hardware and staffing that may or may not materialize.

Doug emphasized that they want to support our needs and he encouraged people to contact him to see how those may best be met.

Dan Cromer mentioned that the other factor in all this is cost. Fedro Zazueta is reluctant to offer programs widely without some method of cost recovery, and that will require negotiations between Academic Technology and the IFAS Dean for Extension. The dual problems of non-Gatorlink authentication and the need for on-line payment from clients are other sticking points in all of this.

Course advertisement

Dennis Brown raised the issue of professors needing to advertise their courses. With Vista 3, a course cannot be accessed until one in enrolled. That does not permit use of the system for advertising courses to prospective students. Doug responded that the Campus Edition of WebCT had a publicly accessible "Welcome Page" for each course, but that this feature was dropped for some reason in Vista. Doug believes that this has been reinstated in Vista 4. If so, that should address this particular concern.

Instructional Design

Trish Capps at the WFREC in Milton has noted a need for the availability of improved instructional design details. She mentioned having tried a couple of times to complete the online training within WebCT which covers how to use the various features and functions of Vista. She has no complaints with these excellent materials, but rather notes that her daily work life has interfered with her being able to complete those. Even as far as she has gotten with that, they seem mainly to cover the various components and how to work with them, however. What she sees as needed is some basic design information to be available on the front-end.

Trish gave the example of their trying to post a narrated PowerPoint. They eventually discovered that Vista handled imbedded audio very poorly (i.e., slowly) and by digging around they eventually arrived at a suggestion that they use Articulate to handle that function. They finagled a license eventually and that worked well, but next they wanted to embed video and found out the hard way that Articulate doesn't support that. Her point was that there is need for some readily available assistance which would help with such course design aspects. Without that, the faculty can get quickly frustrated by the system "not working" when they had been able to do certain things previously using their own websites.

The other aspect which Trish has seen working against WebCT adoption by her faculty is the matter of the reduced screen size available within the Vista system. The various tools within the Vista interface simply do not provide as large a viewing area as folks have come to expect via delivery from their own web servers.

Doug allowed that there are certainly some aspects that do simply work better or at least more easily on the web rather than via Vista. One of the constraints of Vista is that it is Java based. This creates complexities when you work with other tools that need to integrate with that. There is also the question of what is the best tool to accomplish each individual task? There are certainly cases where content may be better deployed outside of the Vista system, and perhaps linked to it from a Vista account. Streaming video and longer audio files are classic examples. If you upload a video in Vista, Vista handles that as a standard file. Thus, when the student clicks on it, the entire file must be downloaded prior to viewing/listening. That is often simply unacceptable, and LSS as well as IFAS provide streaming media services outside of Vista which may be utilized for these purposes. One of the dangers with Vista is that instructors may start re-conceptualizing their content around the tools it provides rather than looking at what might be the best delivery methods for their particular content. When that happens you end up with the technology driving the instruction rather than the instruction leveraging available technology.

Doug pointed out that Trish's concerns over the details of instructional design are very valid. It is no accident that one can obtain higher degrees in the subject of instructional design. It is an extremely complex topic and a huge field of study unto itself. Doug thought Trish raised an excellent point with the idea of a knowledgebase. They have been considering a FAQ-based "I want to do such-and-such, how can I best do that?" format. Doug said he would get with his staff to see if they could use such methods to address some of Trish's concerns.

Where does Elluminate fit in?

Steve raised the above question. Doug responded that Elluminate is a synchronous communication product that leverages primarily VoIP to enable users at separate locations to talk to each other in the context of an on-line meeting place. Elluminate is being made available via Software Licensing Services on a cost retrieval basis. Elluminate can function as a standalone product whereby students can be invited to an on-line meeting place and utilize microphones ($75 Plantronics headsets highly recommended) to communicate. An instructor can use this to engage the class in vocal remote instruction, with the capability of breaking a group into separate discussion sections which can later come back together for discussion as a whole. Elluminate was chosen both because of its ease of use and its ability to work well over dialup connections. Additionally, our Vista 4 system should have a plug-in installed by the end of this month that will support Elluminate directly from the Vista system. Elluminate supports the pushing out of PowerPoint presentations, desktop and application sharing and a number of other very useful features.

Trish mentioned that she and Rick Faulk have been experimenting with Elluminate via the free trial offered on their website.

Dan Cromer related his understanding that it did not allow for the recording of sessions for later playback, but Doug Johnson said that he believed it actually did.

ICC Business

Due to time constraints, Steve attempted to whisk through the rest of the agenda in no more than 10 minutes :-P. The agenda-based structure is reflected below for consistency's sake, but many of the topics were either skipped altogether or touched on in an extremely brief fashion. In order to make actual additions more readily accessible, here is an index to the main topics below for which further information was available, either at the meeting itself or via outside sources:

• Upcoming IFAS IT Policy Advisory Committee meeting
• The UF Exchange Project
• UF Calendar Project
• Virtualization of Core Services
• Vista Issues
• Patching
• Off-line Infection Scanning

Policy

Upcoming IFAS IT Policy Advisory Committee meeting

There will be another ITPAC meeting next Thursday, August 16th. Dan Cromer had a policy matter that he wants to raise there and sought feedback from the ICC on that. There was very little time at the meeting to go into this, but you are all encouraged to contact Dan or Steve with any comments on the issue.

Matt Wilson had raised a question to Dan on whether or not there were any policies regarding database creation and responsibilities. Dan had responded via the ICC-L that:

"The short answer is no, we don't have a published policy that I know about.  
The verbal policy, which is being debated at UF ITAC-DI and CIO level, and 
at some point may be formally codified, is that all databases must have a 
faculty or staff person designated as "data principal", usually at a fairly 
high position of authority, who is responsible for understanding the contents, 
backup/recovery plan, and security of the data stored there. Then we, as 
technical support, are "data custodians" who are responsible for the technical 
implementation of the databases, security, and backups.
  
For now, you can tell anyone requesting a database that they need to designate 
a "data principal" for it, normally the principal investigator (PI) for the 
project. Further, I'd like you to have documentation on each database which 
includes the contact information for the associated data principal. This 
should include Web sites as well, even if they don't have an SQL or other 
database associated; I don't know how far along Mark Ross has progressed in 
getting this done for Web sites, perhaps the same documentation system should 
work for both."

Dan is evolving a system which he hopes can address this issues and similar issues relating to websites. The first step may be to get a policy instituted. He believes working this into the exit procedures would be worthwhile, asking people who are leaving to check off on any websites or databases for which they were responsible. Steve mentioned that some periodic renewal or certification for those might be needed to realistically address the fact that decommissioning of such resources is rarely instituted by the responsible parties.

UF IT Advisory Committee for Network Infrastructure meeting

The ITAC-NI is still on hiatus for the summer, but should resume in September.

IT Governance sub-committee status report

There has been no reported movement on this standing agenda item; it is still pending high-level hiring decisions.

The UF Exchange Project

Dwight Jesseman kindly made himself available via phone, but Steve failed to make timely use of that, having been distracted by the technical difficulties we had with projection and streaming. As a result Dwight didn't get the chance to elicit the feedback he sought on the detailed and accurate flowcharts (ufad\if-admn credentials required for access) he had made documenting the complicated path which messages take from the point they hit the UF SMTP server all the way through until they end up in either the Inbox or the Junk E-mail Folder. These flowcharts, along with other supplemental documentation had been announced via the ICC-L back on July 24th and are now available to provide OU Admins information to best understand, monitor, and influence how spam is handled within our e-mail system.

Please take the time to review these documents and if you have any questions or suggestions for improvement, please let either Steve or Dwight know.

The Wallplate Project

There was no time to devote to this matter at this month's meeting, but Steve is still not convinced that this is the right move for many IFAS departments. No doubt there will be more discussion to follow.

Recommendation: autogroups for *selected* roles

This is generally not discussed (and wasn't this month) but is being kept on the agenda for future consideration. Basic role autogroups are now in place within UFAD.

Split DNS solution for UFAD problems

As a standing issue, Steve would like to remind folks that he is investigating alternate solutions for the split DNS issues. Initial tests look promising.

If anyone else would like to help test this, please get with Steve. Since a split-DNS solution is not expected to happen any time soon, this might really help out with our user experience in the meantime.

UF Calendar Project

Prior UF Calendar Project discussion. Wendy Williams was not given the opportunity to update us on this due to time constraints. It is Steve's understanding that a less ambitious recommendation is being drafted for consideration which includes at this point the following recommendations:

1. That the overall enterprise calendar project continue
2. That the Office of the CIO identify a full time project team with appropriate skills and experience when priorities and schedules allow
3. That this new project team
   1. Evaluate the calendar options identified by the work group
      1. Bedework
      2. University of Nebraska Lincoln
      3. CalDav
   2. Choose one option from those or other solutions
   3. Prepare a development and implementation plan for just calendar.ufl.edu
   4. Prepare a brief communications plan for other calendar administrators and support staff on campus that describes the changes to calendar.ufl.edu

Projects

Listserv confirm settings

As reported previously, this is a low priority work in progress. It has been agreed to by administration, but will take some time to fully implement.

SharePoint Test Site

Prior SharePoint discussion. Ben Beach was not available. Consequently, and due to time constraints, this section of the agenda was skipped.

Virtualization of Core Services

Wayne Hyde wasn't on-hand to provide an update on how the server and storage configuration situation is going. He did, however provide a number of details via personal communication just after the meeting. Those are detailed following:

In order to help solve a number of pressing issues with our server infrastructure, Wayne had developed a plan to obtain a VMware virtual server cluster and SAN to replace IF-SRV-FILE02/03 and other direct-attached storage.

Chris Leopold's IFAS ITSA group is requesting $60,000 per year for the lease of a Dell/EMC Storage Area Network and two enterprise class Dell PowerEdge servers that will result in cost-savings over time for IFAS IT.

While they have been working towards keeping all of our servers under recurring lease agreements, there still are 28 servers that are not under any lease agreement and are scheduled to be out of warranty this year. There are two options to mitigate this issue: these can be replaced by leasing 28 mid-range servers, or we can expand upon our highly successful virtual infrastructure pilot project. Investing in our virtual server infrastructure resolves many of the technical issues facing the server administration group as well as provides long-term cost savings down the road.

The current process for provisioning a new server involves a long process of specifying and ordering hardware, waiting for new hardware to be purchased and delivered, configuring the server, and then final deployment. This entire process may take up to two months. With our current Virtual Infrastructure project, deployment of servers can be accomplished within a few hours as servers can be provisioned with a few mouse clicks. Not only will we be able to virtualize existing servers which need replacement, we can rapidly deploy services requested by IFAS units without purchasing new hardware in many cases.

Some more examples of potential cost savings and improvements in services down the road include:

• Microsoft Operations Manager

  Our MOM cluster, which is used for monitoring our many server resources, cost about $30,000. This includes a back-end SQL server, two front-end servers, and 1.75TB of local storage. Instead of purchasing a new cluster, the entire project can be virtualized. We would need to add an additional server to our Virtual Infrastructure, but we could have a savings of about 33% on this project. The end result is that we improve our virtual cluster with the addition of one more enterprise server and see a cost savings of $10,000 on the budget.

• Highly available services

We can also leverage the ESX cluster to provide highly available services for IFAS. We have critical services such as SQL databases which are hosted on a single server and prone to an outage due to hardware failure. If, for example, a critical part in IF-SRV-SQL02 fails, all of our databases hosted on that server will be down until the hardware can be repaired. By clustering the SQL databases between the physical SQL02 server and a virtual SQL server, we can provide a highly available database cluster for IFAS at the cost of a second SQL server license.

• Web services

Other services can be clustered as well such as our Web server. Too much of our current infrastructure has single point of failures with the only redundant service currently in IFAS being our file servers.

• Hardware consolidation and flexibility

The virtual infrastructure has also helped us in a few cases where we had dying server hardware and no funds to replace the servers with new Dell servers. One example is IF-SRV-GIS. The hard disks and raid controller were failing, so the server was converted to a virtual server using the Vmware Converter software we purchased. The process took about an hour to complete and allowed us to remove the old hardware from service without a long interruption of the services provided by this server. The other benefit realized by this conversion was that the new virtualized server performed much better than the original physical server. Two other servers have been virtualized this way including our WSUS server which provides system updates to all IFAS machines and another departmental server for SWS.

• IFAS Content Management project

An example of a future cost savings is the IFAS Content Management project. We will be able to leverage this new virtual infrastructure to reduce the hardware costs of the MOSS project by up to 75% by virtualizing three of the five proposed servers and tapping into the new Storage Area Network for storage space.

These are only a few of the many examples of how VMware’s Virtual Infrastructure will greatly benefit ITSA and IFAS.

The details of the plan include getting a few additional PowerEdge 2900's with 48GB RAM and dual quad-core processors all tied to a new SAN. Hardware would get swapped around so the MD1000 is on IF-SRV-BACKUP for our backup-to-disk-to-tape routine. The 2900's would host virtualized servers like we currently do (wsus, file03, etc) but on a much larger scale. The SAN will also house the network file storage for IFAS users, our IF-SRV-FILE02 server will go back to being primary, and we'll eliminate the DFS data replication scheme which has proven problematic.

VMmware has some "high availability" functionality so if one of the big-iron boxes fails, the virtualized servers would get restarted on the other servers in the cluster. It will look like the server crashed, but it will return to service after a quick reboot. It also has load balancing so it can move running virtual servers around so the balance stays even between the servers.

This move will also allow us to restore shadow copy on our file servers. Once we have the SAN we'll have a boat load of storage for shadow copy space, so endusers will be able to do many/most file restores all on their own. That is a big selling point for our centralized file services that has needed to be held off on due to present limitations. The current direct storage on IF-SRVFILE02 will provide shadow copy storage and the MD1000 on IF-SRV-FILE03 will provide storage for backup-to-disk. Backups of the virtual servers get dumped to disk, then to tape. Restores should be done from disk, as we'll have enough retention time on backups. Tapes will be for when it hits the fan. We'll also re-do the file server storage so there isn't just one big 7TB drive. Units will get split up on smaller disks; for example, PLP will get its own 2TB volume (they use about 1TB now), etc. This configuration will speed up recovery if a single raid volume dies or if file checks need to run. Chkdsk on a 7TB volume can take an entire day, which is bad.

Paying for these robust but necessary services may require asking departments to contribute; however, the details of that aspect are still pending consideration by administration. If you believe that your department would benefit from such a move you are encouraged to seek support at the unit level; we must all work together to get the necessary support for our core services.

IFAS WebDAV implementation

Still no movement has occurred in getting this documented.

Vista TAP and Vista Deployment via SMS and WDS

There was no real time to devote to this topic this month. Steve did note, however that Dennis Brown had found a Vista issue regarding wireless authentication. Steve has started a "Vista Issues" section of the ITSA Service documentation (ufad\if-admn credentials required) and Dennis has the honor of being the first entry. This section was added under Desktop Deployment > Vista Issues.

New IFAS IP Plan

No confirmation yet, but this project is believed to be completed--or nearly so.

Exit processes, NMB and permission removal

Prior exit procedure discussion. Progress on this is still pending and time did not permit going into any details this month.

Operations

Patching

Obviously, we are coming up on another "Patch Tuesday" next week. There are expected to be six critical and three important updates covering various versions of Windows, Office (including for the Mac), IE, and Microsoft's Virtual PC and Virtual Server Products.

Off-line Infection Scanning

Wayne Hyde has developed some off-line scanning tools for general use. Steve wanted everyone aware of these. Documentation is now available (ufad\if-admn credentials required).

The meeting was adjourned early, at about 11:50 am, in order to make room for another meeting which was due to begin at noon.