ICC Meeting:

IFAS COMPUTER COORDINATORS
(ICC)

NOTES FROM January 14th 2011 REGULAR MEETING

A meeting of the ICC was held on Friday, January 14th, 2011 in the ICS conference room. The meeting was chaired and called to order by Steve Lasley at about 10:00 am.

PRESENT: Twenty-three members participated.
 
Remote participants: Tom Barnash, David Bauldree, Bill Black, Micah Bolen, Dan Christophy, Dan Cromer, Dean Delker, Francis Ferguson, Kevin Hill, Scott Owens, Joel Parlin, Nick Smith, and John Wells.
 
On-site participants: Dennis Brown, Andrew Carey, Wayne Hyde, Chris Leopold, Francis Ferguson, Winnie Lante, Steve Lasley, Nick Smith, Santos Soler, and Wendy Williams.
 
Guests: Iain Moffat and Elias Eldayrie
 

STREAMING AUDIO: available here

NOTES:

Agendas were distributed and the sign-up sheet was passed around.

Report from the chairman

Member news:

The only membership change Steve had to report was the replacement of Jin Wang by Phillip Ho working under Wendy Williams at CALS.

Recap since last meeting:

As per his usual procedure, Steve pointed folks to the notes of the last meeting, without going into any details

Special guests: Iain Moffat and Elias Eldayrie

Introductions

Once Iain and Elias arrived, Steve took the time for introductions around the room and across the state. He then explained that our guests were here today due to his having invited Iain originally a couple of months ago to discuss the incorporation of what was formerly called the Microsoft Applications Group under his supervision at OSG within CNS. Iain was very receptive to this but was unable to schedule it until now; in the meantime, Elias had heard of our concerns and wanted to have the opportunity to address them as well.

Why the MAG group was moved into OSG at CNS

Steve introduced the topic by saying we hoped to begin a discussion about how the MAG move happened and why, along with how it has been progressing. The ICC is looking to engage OSG in building a relationship with this new organization--an organization which provides many of the centralized services most important to us and our users.

Assessing IT at UF

Elias responded by saying that upon arriving at UF he began assessing our situation by looking primarily at three factors: the technology platforms we were using, our IT processes, and the people we had for running those. The last few months have been spent figuring out where we are and, most importantly, where we want to go. Elias said that he focuses much more on how we are going to get to where we want to go rather than where we have been. Historical events are important, but they will not dictate where we will go.

A glaring organizational issue

One of the glaring issues Elias saw was in the "Microsoft Applications Group". First of all, he hated the name; he doesn't feel organizations should be named after companies. He believes that, for the most part, the MAG has done a very good job servicing UF with very limited resources. The more he looked into things, the more he saw that UF had mission critical systems being supported for the most part by a group that had no backup. They were one man deep with a single point of failure in a number of different places and there was no good integration between them and other services within UF. Considering the dependency of other services on those platforms, authentication among others, the relationship and integration there was pretty strained.

In danger from single point failures

Elias said that he had only been here two or three weeks when Dwight Jesseman decided to leave UF, someone whom Elias understood was probably one of the best in the country in terms of his skill set. People started coming to Elias, basically freaking out, saying the world was ending and asking what we were going to do. While all that may be true, it dawned on Elias that UF really ought not to be in a position where we are at that high of a risk just from one person leaving.

Reorganizing for robustness

This reconfirmed Elias's thinking that UF needed to move the MAG group into a place where we could begin to build redundancies, business continuity, personnel backup, and begin to develop skills that other people may not have but should have. But while doing that, the services needed to be protected as campus very much depends on these services to function.

Aligning UF with practices at other schools

Organizationally, Elias did not see the need to have a small group reporting directly to him. If you look around the country at the way such things are structured elsewhere you will find that services such as those provided formerly by the MAG group really are housed where we have them right now.

Putting the past behind us

Elias wants to put the past behind us and focus now on getting together people who want to be on this team, play nicely, and serve this university. Elias believes UF is a great place or he wouldn't have come here; he was not looking for a job when this opportunity arose. This is a wonderful place and he wanted to be part of it. We need to make sure that the IT organization is set up in such a way that it supports UF's mission; then when Dwight, Iain, Elias or any of us leave we will have a nice farewell party for you but the university will continue to do its business as normal.

Engagement from campus will be proactively sought

So, Elias has moved the organization to where he thinks it belongs, where common services can leverage each other. Obviously, some of the ICC were concerned with this transition and Elias has heard a good deal of discussion about how the decision was made, who was being consulted, etc. Elias said that he will not be walking around campus consulting everybody about every decision he will be making--especially on organizational issues. IT will be engaging the campus very proactively about what we will be working on, however, providing a portfolio of the services we provide, asking if we are doing things rightly or wrongly, and if we should sunset some services or build new ones, etc. We will be doing that very actively through a governance which is now in place. Elias will engage the university very proactively on our portfolio of services and our strategic direction.

Organizational structure and processes will be under Elias's purview

Elias made it abundantly clear, however, that the organizational structure and the processes of how we run these things will very much be within his own portfolio. That is Elias's plan and if he wants to reorganize things then he will do that as long as those decisions are made for the greater good of the university.

Iain is looking for better ways to engage the ICC and other campus IT groups

The MAG group has now been moved into CNS and he has heard concerns from some in the room as well as others about the services or lack thereof which CNS provides. His answer to everyone is that he wants to know what the problems are and they will fix them. Tell the people that are in charge to fix it and if they don't because of some irrational reasoning, then Elias will get involved. Elias said that Iain is a great guy who is trying hard to figure out the best way to engage the university and specifically the ICC and other such groups on campus. He wants to provide some comfort level that the services we provide will continue and are very important to us. They are not going out of their way to break them; the last thing Elias wants is for services to break.

Weaknesses in MAG group services being located and fixed

Already, as OSG is getting into the architecture of how the systems were set up, they are beginning to find some weaknesses which Elias believes require mitigation. OSG is working on those to minimize downtime, to ensure that when things break we have a reasonable response time. Some very important services and mission critical systems were not even being monitored by that previous group. So Elias is trying to make sure these systems are being looked at holistically and ensure that they are integrated with other services which the university provides. To do that, you have to have these groups together.

How is the transition progressing?

Steve asked if the discussion could now turn to how things are working currently. He admitted to not knowing the OSG very well and was not clear about how the MAG group was being incorporated. Steve asked Iain if he was forming teams around the various services or beginning to cross-train his staff.

James Oulman has been learning Exchange

Iain responded that they have indeed been grouping "like" people. Ever since Dwight, left James Oulman has been working very closely with Luis Molina, our lead Exchange person from the MAG group. They have been planning the Exchange 2010 implementation and have been doing a stellar job. James had previously been doing a lot of other work within OSG, but is now refocused and has come a long way with Exchange and Active Directory, which are very key services.

Increasing storage capacity has been one challenge

Among other things, they have been working on a lot of the storage architecture. When they first looked a few months ago at what was needed to transition from Exchange 2007 to 2010 they noted we lacked sufficient storage. With Elias's support they have dropped another $90K into storage for Exchange 2010. We have that now and James has been heading up getting all that installed and getting us more rack space so we have the necessary resources ready.

SharePoint

Buck Buchholz is the SharePoint point person from the MAG group. Buck has generally lacked support; he has Jason Heeter, but Jason is in a time-limited position and doesn't necessarily have the infrastructure skill set that Buck enjoys. Consequently, Keith Sanders, who works for Iain's team and does a lot of their SQL work is now being trained part-time on SharePoint.

Building in continuation of service

Everywhere it is applicable Iain is pairing up people with the goal of providing personnel backup and continuity of service. Individuals should be able to take vacations and not worry that things will crumble in their absence.

Development of an OSG ticketing system

Chris Leopold said that he has been pushing pretty hard on his director, Dan Cromer, to get some sort of information conduit created with OSG so that when issues arise we can get those addressed efficiently. Chris has known Mike Kanofsky, who is leaving, for thirty years; up until now he has been able to pick up the cell phone and get direct access to Mike when problems arose. While Chris realizes this might not be appropriate for an enterprise environment, this is what has worked in the past and he is looking for a suitable replacement mechanism.

Iain responded that Dan and he had been talking about that just the other day. Iain realizes that such things are changing for IFAS now that Mike is leaving UF. Iain admitted that OSG has not really broadly used a ticketing system in the past but he related that they are now working on a project to improve that. It is his hope that OSG can provide IFAS an easy interface for submitting tickets so that issues might be tracked and that IFAS can get the response they require. OSG then should be able to use system reporting to assure their response time remains acceptable. Iain said he would appreciate feedback either way on whether or not this is the sort of thing IFAS is looking for.

Reviewing the UFAD SLA

Chris Leopold mentioned that there is an SLA which had been set up with UFAD; he suggested that we sit down and review those expectations looking ahead and Iain agreed.

Ongoing meetings involve Dan Cromer with both Elias and Iain

Dan Cromer wanted to make all aware that he meets personally with Elias every two weeks and he attends a meeting with Iain and his team every Monday morning. Dan wanted to reassure the ICC that he is having ongoing conversations with everyone; if you let him know your problems and concerns he will raise them to the people in charge of our central services. He mentioned having had good discussions with Iain and would like to compliment him on his openness and responsiveness.

Dan expressed his preference that issues be discussed first here with the ICC so that everyone is aware of those and can discuss our needs among ourselves. He would rather not have individual ICCers calling Iain directly but would rather funnel things through the ICC and himself. Dan feels that most issues then can be addressed directly between OSG and himself; more complicated issues might then involve Elias.

How will the replacement of Mike Kanofsky be handled?

Andrew Carey asked about the plans for the UFAD service now that Mike Kanofsky is leaving. Andrew noted there is considerable concern across IFAS and UF in that regard. He wanted to know if anyone is cross-training with Mike currently.

Mike says goodbye

As background, we all learned the sad news when Mike posted the following to the CCC list:

Determining what tasks need transitioning

Iain responded that the immediate issue is determining exactly what things Mike has his hands in--and that is lot of stuff as we all know. James has begun working on a lot of that and Luis Molina has considerable expertise in Active Directory. Erik Schmidt also has a lot of UFAD expertise. OSG has different people working on the various areas and feels they do have the personnel to step into those roles until a replacement is located. They do need, however, to re-engage those across campus who have been working directly with Mike.

Preparations are complete for posting

Near to mid-term, Iain has developed a new updated job description for Mike's position; that hadn't been updated since Mike moved into UFAD about seven years ago. That description is now more accurate and includes such things as OCS; it now covers those aspects in which we will want a new candidate to excel. The vacant position will be posted any day now and will be advertized on Monster, The Chronicle of Higher Education, Educause, and a couple of other places including Windows Hied. Iain wants to make sure he gets the attention of those individuals out there who may be a good fit for us.

Has OSG considered leaning on expertise elsewhere at UF?

Ben Beach asked if Iain had considered branching out to other departments to tap the considerable expertise available right here at UF. He suggested that some cooperative relationships might provide additional support and expertise that would be useful to OSG. Iain said that this was not something that had been considered but that he could see where such a thing might be of use.

How urgent is the need to transition to a 2008 R2 native domain?

Kevin Hill related that prior to Iain's arrival today there had been a discussion about a rush need for finishing Domain Controller deployment prior to Mike leaving. The thought was that we need to retire older DCs immediately so we can move UFAD to a Windows 2008 R2 native domain. The transition to Lync is just one reason that must happen, but Kevin wanted to know if the urgency was real or if that could be delayed somewhat in order to provide him more time.

Iain encouraged Kevin to move forward with the deployment as we would like to get that upgrade done as quickly as possible. Iain would like to get as much of that project wrapped up as possible before Mike leaves; if not, of course they will continue on after. Iain definitely sees no reason to wait, however. Kevin responded that he would be more comfortable waiting until Mike's replacement was in place before making this important transition. It is not that Kevin is trying to delay things, rather he is concerned that Mike's leaving should cause an "emergency" need to rush matters.

What is the timeframe for filling the position and what is the approach?

Kevin asked what timeframe was expected on filling Mike's position and whether we were looking to fill that from inside or from without. Iain was completely open in saying that James Oulman is an internal candidate. James is very interested in moving into a full Windows, Exchange and Active Directory role. He has a vast amount of expertise there and Luis has been encouraging him to apply for the position.

If that happens, Iain's full intent is to rewrite James's position to be Active Directory and Exchange and repost that. Iain mentioned this because he doesn't want people to think OSG would be a person down should James take the position. That will not be the case as we would still seek another hire with this same skill set. Either of those can basically be pursued in parallel as they already have rewrites for both positions and he expects those to be posted within the next few days.

Iain said that while he will proceed as fast as he can, we all understand that the hiring process will take a while. Iain said that he believes Kevin's concern related to that gap; OSG has both Luis and James to help fill that. Keith can also assist, not necessarily with the nitty-gritty DC work, but with a lot of the day-to-day work that comes in. Erik is helping as well and Iain feels we have a lot of good team members to cover this. The projects that Mike is working on may not proceed as quickly because new contexts will have to be learned by various individuals. We can keep working on things; they just may initially take a bit longer than prior.

More staff but also more duties: will this move be synergistic?

Steve commented that Iain has more staff now plus a whole lot more things to manage. Steve asked if Iain felt this was a winning situation overall for OSG and that a synergy might form from this integration. Iain believes there is a great synergy there. He gave the example of their extensive virtual hosting services where Microsoft-focused hosting provides greater revenue than that for other systems. OSG has huge customers in the library and DCE for Windows file sharing services (20TB for the library) and MS SQL is really huge as well. James and Keith and others have been doing that for a while already and are used to working in that area.

Iain said that things are working the other way as well. Luis expressed interest in learning more about Proofpoint so that when something comes in he can get an end-to-end view of it and not have to hand it off to someone else.

Are the OSG file hosting solutions appliance- or Windows 2008-based?

Chris Leopold asked if the file services being hosted for the library were on an appliance box or if it was actually a Windows clustered environment. Iain responded that OSG actually offers both, but the library chose to utilize OSG's EMC storage appliance which is integrated with UFAD. That appliance lacks a minimal feature subset, notably DFSR, but the library did not require that; they were more interested in the redundancy provided. One unit is housed at SSRB and another at CSE. Each consists of three blades which equate to really large servers at each spot so it is triple redundant at the box within each site and if there is a site failure it will pick it up at the other site.

Naturally, OSG can accomplish a very similar thing using Windows clustering. OSG has other customers that want Windows Server 2008 R2 machines and they tailor offerings to that as needed.

How will continued innovation around Microsoft products be addressed?

Chris Leopold also inquired about how future Microsoft-based service offerings might considered and handled. Within IFAS we discuss needs here at the ICC and then raise policy decisions to ITPAC for consideration as appropriate. When directions have been set, then the IFAS IT Server Administration group handles the implementation. Moving ahead, should IFAS decide that some new service is needed--say Direct Access as a VPN replacement--how might that best be raised to OSG for consideration? Should these things be brought to Dan Cromer and then from him to Iain or Elias?

Elias quickly responded that he does not expect to be involved with decisions on that level. Rather he has encouraged Dan to contact the service providers directly for resolution; Elias feels that Dan is very good at that. He said that he gets involved only if people are not playing nicely with each other or if we need to make an investment for which we have no resources.

Access to decisions and decision making

The new IT governance structure

Elias then briefly described the new governance organization and passed around handouts of its overall structure. The governance has a "Shared IT Infrastructure" group where Elias encourages everyone to bring whatever issues we believe the organization should address. Elias would like to leverage that governance wherever possible.

The Campus IT Directors group

Elias has also established a group called the "Campus IT Directors", of which Dan Cromer and Wendy Williams are currently IFAS members. That group meets now on a monthly basis and is also very much encouraged to bring whatever issues they think are important for group discussion. There are a number of different venues which have been established for people to communicate back and forth with the IT Organization to ensure that the services they provide meet expectations.

Hopes for more rapid adoption of innovation around Microsoft solutions

Steve pointed out that in the past a number of innovations at UF began as grassroots movements here at IFAS; getting from there to centrally-based services was difficult but ultimately successful. Steve is excited that the opportunity for a strong centralized group now exists with experience in the Microsoft solutions upon which IFAS has built and depends. He hopes this might lead to centrally supported implementations much more quickly in the future. One example is the current interest in SCCM; it would be great to involve OSG in that as early and often as is possible.

OSG team meetings will be open

Steve asked if there would be other conduits of information available other than through the committee and organizational structure. Are or will there be open meetings that interested parties might attend to keep abreast of current OSG projects? Iain responded that OSG has team meetings on Monday out at the East Campus which everyone is welcome to attend either in person or via LiveMeeting. Steve said that sounded very promising, as the ICC has interest in building new relationships in whatever ways we can.

How will transparency be handled with the topical committees?

Elias asked Steve if he had any ideas on how UF IT might better engage the ICC than how they are doing currently. Steve responded that he had heard the term "transparent" used in referring to the new governance structure. One of the things Steve feels is necessary for that is to be able to see what these various committees are doing, which Steve believes would entail the keeping of good meeting minutes.

Steve asked if it is the plan to have a secretary for each of these committees that will publish what is going on or is that outside of our means? Elias responded that we cannot afford secretaries and asked that Steve please not suggest that to the groups. We are, however, asking them to keep notes; we can't have minutes because then it becomes a public record conversation which we are trying to avoid. You have to invite the media and there are legal terms which we just cannot use according to the UF General Counsel--though Elias admitted he was not clear on the entire context behind that. However, notes will be kept for these meetings and those notes will be posted on the web site for each of these groups. That should provide an idea of what the conversation is all about and what the issues are which are being brought forward.

A web-based forum for input to the committees is planned

Ben Beach asked if that web site would include a forum where any IT worker might be able to make comments or suggestions? Elias said that yes, there will be an opportunity for people to provide information electronically to those committees and for them to take it under consideration.

IT staffers are likely to be caught by surprise from time to time

Steve pointed out that what has tended to happen in the past is that something is decided and we are the last to hear about it, even though we are the ones who have to implement it. Elias responded that this will very likely happen again. This is a large place and you are not going to be able to touch base with every single person all the time in the right sequence. There are going to be times where you find out something that you probably should have found out earlier. The scale and sequences within which events have to take place sometimes means things just have to go.

Elias said he is not going to pretend that the governance structure will solve all the problems; rather this is going to provide a venue for us to talk as collaboratively across the campus as we possibly can. Elias added that he is open to adjusting things as is appropriate as we move forward. If people have different and better ways of accomplishing things, Elias is all for it.

An example surprise notice: the Electronic Copy - Print Output Cost Reduction proposal

Steve asked if we could use as an example the "Electronic Copy - Print Output Cost Reduction" program which just became public yesterday. The notice forwarded to our Departmental Chairs and Directors from the deans along with Dan Cromer's forward to the ICC-L were basically the first any of us had heard of this. It seems to be something that is a mandatory thing which will be quite sweeping in scope. Should we expect things like that often in the future or was this surprise the result of its arrival ahead of governance development?

This was not an IT initiative

Elias responded that this is not an IT initiative; he takes the blame for many things, but this is not one of those. Elias explained how this proposed project happened. University leadership has the obligation to engage our units when such issues come to the table. A presentation on this initiative from the CFO was made to the President's Cabinet and all the Senior VPs and VPs were present.

A year and a half ago, before Elias came here, the university engaged a consultant to inventory how much printing we do and how many copiers/printers/all-in-one type devices we had. That work was done by the Procurement Office. The consultant came back and made a recommendation about a different model to manage printing on campus. That model suggests that, if we change things, the university will save $3 million. That presentation was made to the senior leadership.

It is Elias's job to say, wait a minute--I can't sign off on this yet until I go talk to my staff. It is not the problem of the person who presented this, rather it is Elias's job to either get feedback from his staff to see what this might take or to say he knows enough, it is fine, and he will just sell it to his staff. That's how things occur.

This proposal was not mishandled

Elias doesn't believe this was really done the wrong way. The VPs who are responsible for their areas should make the call as to how much they want to engage in these decisions with those they represent. The President looked at it and asked if everyone was okay with it and they all said we are good to go. UF is a big place. If you want to make a decision and you have to talk to every group, committee, and person on this campus then nothing would ever get done. We all know this from having been on committees where a lot of discussion occurs but no decisions are made; that is higher education for you.

Understanding higher education

Elias sees this all the time; sometimes thirty-six to one is a tie in higher education. You hear everybody, they all express their opinion; you trust the leadership that they are going to be held accountable for the decisions they make. If they work, great; if they don't then someone will have to do something about it. So, that is how that decision was made and Elias is not sure how else you would do it. If you are the CFO and made that presentation, what else would you do?

Ben Beach said that they might include someone from IT. If it was anything that could have an impact on IT Ben would want to have someone from IT at that meeting. Elias pointed out that he was there at that presentation last week.

The message may have been misinterpreted

The message that came out of the meeting was that it is being implemented and all the details have been figured out; that, however, is not what was presented. What was presented is a model which provides the opportunity to save $3 million. We still have to have an Invitation to Negotiate (ITN) for someone to come on campus and really prove that.

An ITN will work out the details

Elias commented that we all have dealt with consultants; they sell you everything but what they need to be selling you. Elias told Dr. Machen after the meeting that taking that $3 million and dividing by some number--that is probably closer to the truth. But in his defense, Machen said there is another process which we will go through. We will do an ITN; someone will come on campus and look at all the details of location, networking, support, etc. The only thing which was agreed to was a model to, for the most part, outsource printing. The university will move to a model where we will no longer own these big copiers. They will be provided by the vendor and instead we will be paying by the click.

The rationale behind the proposed new model

According to the consultant that did this work prior to Elias's arrival, the average cost per print on this campus is $.076. The consultant is saying they could bring that cost down to $.016. That's a lot of money, so if you are the President of the university, this is something about which you would wish to hear more details. That's where we are. Elias said that other important point is that the money being saved under this proposal would stay with the units. The university is not trying to grab that and retain it centrally. Looking at it that way, Elias believes this is worth looking into further; let someone come in and demonstrate these savings are feasible. Elias believes that at that time there will be more engagement about how we are actually going to move on to supporting it.

Budgets are a continuing worry and cost cutting will remain an important topic

That is how these things happen. Will more of these happen in the future? Probably. The university is trying to see right now if we have an immediate budget problem. At every cabinet meeting people are waiting to hear whether or not we have a budget cut coming. Everybody is going to be looking around to see where we can save money while continue to provide services.

What will be the scope of this new printing model?

Wendy Williams asked if this printing initiative would include computer labs where students print. Elias responded that he did not know the answer to that. This is one of the questions which was followed up on the next day by Fedro Zazueta. Lisa Deal with Procurement was the person who really coordinated with the consultants and Fedro has a meeting scheduled with her for next week to see exactly what is in that $3 million.

Dennis Brown asked if this program was meant to cover not just copiers but also individuals' printers as well. Elias did not believe this includes desktop printers. He thinks rather this refers to the "big devices" out in the units. Dennis then asked who the vender was and Elias explained that we don't have one. The next step is to do an ITN and select a vendor.

Wayne Hyde asked if the savings possible from replacing individual printers with shared network printers had been investigated. Elias said that the idea is to eliminate as many of those individual printers as we can, but he has been in high education long enough to know that a lot of faculty are not going to like this. Elias believes they are going to try to go after the "big rocks" with this.

As an example, Elias mentioned that he doesn't have a printer in his office and hasn't had for about twelve years--retrieving output is the only exercise he gets. Some people are not going to want this but there are other people really in need of special services: the College of Architecture and other units where they need plotters and other devices. This will not be a one size fits all deal. The ITN process will dig in to flesh out those details. This will give UF the opportunity to invite vendors and talk things out with them before deciding which way to go.

This program is expected to be limited to the Gainesville campus

In response to another question from Dennis, Kevin Hill reported that a qualification came out yesterday that this proposal would be limited to the local Gainesville campus.

Elias did not know the answer to that but said there will be a committee identified to work on the ITN and he knows IFAS will be a part of that.

Summary on the printer topic as an example collaborative process

So those are the details on the printer proposal. The good news about governance is that it creates a venue for participation; the bad news about governance is that it can become bureaucratic. Consequently, some sort of a balance must be struck. Those processes will have to be evolved and the risks must be managed.

Like everything in life, nothing is 100%. If it is 80%, then Elias is going to be a happy guy; but if it is 20% then obviously it doesn't work. There will be things that are going to be missed and decisions will be made that people will find out about afterward. We have to accept those things for what they are and manage them.

Elias reminded us that the overwhelming majority of such proposals come from people with good intentions. They may overlook something because their role is not as broad, but they do have good intentions. In Elias's experience, one is able to talk to those 99% with the good intentions and they will be willing to re-evaluate in the light of additional evidence. Of course, there is the other 1% and you must deal with them as well. That's how it's going to be and that is what we must work through to get things done.

The balance between diversity and centralization

Dennis Brown said that he wanted to obtain a better idea of what direction the campus might be heading technology-wise. He mentioned that IFAS has been very Microsoft-centered and he was curious to know what trends Elias saw throughout the rest of campus. Elias responded that nearly every technology you can imagine exists somewhere on the UF campus. UF is very diverse and, rightly or wrongly, very decentralized. On the Enterprise side, Elias said that we are trying to be as diverse as we possibly can while still be able to support all these things. Elias is seeking some type of balance between diversity and centralization where some technology will be standardized--but it is not going to be every single technology. We will still have to support innovations and meet the emerging needs of our user base.

Elias is not married to a particular technology

Right now we have a pretty diverse portfolio. As we move forward, with every initiative we adopt we will evaluate which technology best meets the needs. Elias is not religious on any technology; he is not a Microsoft guy or Apple or Sun. He has never been that way, does not believe in that way, and doesn't think any organization should be that way. He thinks we should look at each problem and evaluate the best solution. If that is Microsoft, so be it; if it is not; so be it. We will be as diverse as we possibly can as long as we can still support it.

Engaging vendors in roadmap conversations

Elias said that one of the things they are going to do is engage our major vendors in strategic conversations. This afternoon, for example, Microsoft is going to be in Elias's office for two hours giving him a quick look at their roadmap. To the extent it doesn't turn out to be a sales pitch, we will probably open up that conversation to larger audiences on campus. Dell was here yesterday and Oracle is scheduled soon to provide a roadmap conversation. All of that is being done in the hope of being engaged with where the technology is going and how we position ourselves with those roadmaps.

About innovation

Elias is well aware that a lot of innovation doesn't necessarily happen within centralized organizations. Centralized organizations are built to standardize and leverage economies of scale. That allows for the scaling of systems and provides insurance that they are robust. Once in a while central organizations are innovative, but in general, innovation occurs out in the units.

The Active Directory conversation is a very good example of grassroots effort where a service was evaluated and its potential for widespread usefulness became apparent. Let's throw out weight behind that and make it happen. To Elias, that is where a central organization provides value; it provides a means of supporting such things and making them reality.

But you obviously cannot do them all, so you pick and choose where they fit into the roadmap of the university and the roadmap of where those technologies are going.

Summarization and final discussion

Elias summarized this discussion by answering the question, are you guys going to find out sometimes that a decision has been made and you didn't know about it? Yes, you are. This happens even to Elias who sits at the highest levels of the university. Then you find out that these people only have eight hours in the day and have seven thousand people to talk to; it's okay. With good intentions you move on. Elias will engage the people as well as our major partners in strategic conversation and that process has already begun.

Management style

Steve thanked both of our guests and commented good naturedly that he had been a little concerned back when Elias mentioned getting involved only when folks aren't getting along. Steve hoped that this wasn't Elias's main motivation for being here today. Elias responded that he really believes in hiring good people and then getting out of their way. Iain does not need Elias; what Iain knows Elias will never learn and honestly does not want to learn. That is what Elias had meant; he tries to provide a supporting role in helping them do their jobs. He truly believes leadership is not about having people follow you as much as it is having people believe in what you are trying to do and doing it together.

Elias is not looking to make every decision and he is not looking to be every place. These people know what they are doing and are pretty good at what they do. If they need his help he will help them; if not, he will get out of their way. That is the way he likes it to be and he tells people all the time: if you come to him and complain about each other then one of you is not doing your job; if so, he will have to do your job. He tries to minimize that so (joking) nobody wants to come to his office.

He also tells people to pick up the phone and call somebody. IT people are very good at drafting e-mails. You hit the send button and think, well, I spoke my mind...you didn't hear me... Elias said that we all know that e-mail is the worst form of communication if you want to solve any problem. Consequently, Elias has a "two e-mail" rule. If a problem doesn't get solved within two e-mails he picks up the phone. He expects us to pick up the phone and talk to him and he encourages his people to do the same thing. If you have a problem with any of the services, call the provider and ask what is going on--as opposed to deferring to some hierarchy for the issue to get resolved. Elias tries to get out of the way of his staff and he suspects they like it that way.

The IT org chart

Elias also provided an organizational chart to help us visualize the organization that he is building. There is one change not noted: Brian Beach has decided to take on a new responsibility outside UF and is returning to the private sector. Brian is the Senior VP who is Elias's boss until January 25th. After that, Elias will report directly to the President as Brian's position will not be replaced.

Other than that, this chart is what we look like today; how things will look next week Elias does not know.

Topical committees have begun to meet

Steve had noticed that the chairmen of the various topical committees have now been posted and asked if they have started to meet. Elias responded that three of the six have met already for the first time. Education and Outreach has met, Research Computing has not. Administrative Systems and Web Services have each met but Security and Shared Infrastructure have not.

Steve mentioned that he hasn't heard much yet from our new IT Security and Compliance head, Rob Adams. Steve asked about how he is getting along. Elias responded that Rob is still trying to find his way around campus; Elias knows from experience that it is like drinking from fire hose at first. There is a lot of stuff coming your way; it is a new culture and a new system that requires considerable adapting. While attempting to learn that, Rob has been thrown into the middle of managing a Federal subpoena served to UF investigating the manner in which research funds have been managed here. This is an enormous lawsuit similar to what Duke and others have already gone through. Rob is representing us currently with the General Counsel; the good news is that he just finished doing this with Duke so he has the experience to do that here.

Elias encouraged the ICC to invite Rob to an upcoming meeting; he is sure Rob would be more than happy to do so. Iain said that Rob is a really good guy and very personable. Iain's team has actually been working with him a lot on this investigation. He knows that Ron and his team are swamped with that because of all the information which is being asked for.

Thank yous

Steve thanked our guests and mentioned having met Elias prior at an ITPAC, but that this was his first opportunity to meet Iain; for some reason their paths had never crossed before but he was very glad to have finally made his acquaintance. Steve mentioned that he had been in contact with James Oulman for the first time recently concerning the transition from Barracuda to Proofpoint and had been very impressed with James's openness and responsiveness. Iain responded that James is a very good guy and OSG feels lucky to have him. Iain mentioned it has really been fun to watch James and Luis work together. Luis kind of picked up our Exchange 2010 infrastructure from Dwight and Luis had some slight changes to suggest on the architecture. Then as James dug into it more he asked why are we doing these double proxies and these other things. It has been fun watching them figure out how to streamline things.

Elias thanked the ICC for giving them some time this morning and he was sure we would meet again various places on campus. In the meantime, he asked that we let them know and keep them engaged. Elias and Iain then left.

After thoughts from the discussion with Elias and Iain

Concerns remain

Kevin Hill wanted to note that his concerns remain about OSG favoring open source solutions over Microsoft solutions. For Kevin, CNS and OSG have always been sort of a "black hole". He doesn't know people there and the IT/SA group doesn't know that whole team yet. Kevin is concerned that access to the solution providers will be all too cumbersome and he fears that Remedy ("which takes two minutes to load and then you don't know what the heck to do with it") or some similar interface will not be conducive the sort of collaboration that leads to rapid problem solutions.

Kevin added that communication is the key and the relationship with the folks supporting us is absolutely critical. It is important to tier 3 support to be able to feel they have a handle on their environment and be able to deal with an issue without having to send in a ticket and have to wonder who is looking at it and when they will look at it.

Patience and cooperation will pay off as we learn new ways to collaborate

Chris Leopold responded that he shared Kevin's sentiment, but suggested that we first give OSG a chance to get things organized regarding this transition before we lay on the criticism.

Dennis Brown noted that OSG does have an email address (open-systems-l@lists.ufl.edu) and that is generally how he contacts them with good results. A forms-based contact method is also available. Steve would like to note that the "UFAD Team" distribution group (support@ad.ufl.edu) has added a number of OSG folks now as well, including James, Keith, and Iain. Dennis and Ben also pointed out that Elias has urged us to pick up the phone as well.

Steve noted that Iain has expressed the desire to engage with us and he felt that was further indicated by his willingness to be here today. We have started a conversation and Steve tried to set expectations that it could actually be great to have stronger centralized Microsoft support. Steve reminded folks we have been saying for years that the MAG group didn't have enough people or support from central administration. This is just the first step and it will be an evolving process which we will have to continue to work at and improve. Steve reminded folks that this move has stressed OSG nearly as much as it has the MAG group, but they are clearly taking steps to deal with it and they deserve our patience and assistance in working things out.

Concerns with CNS not limited to OSG

Kevin provided the example of the trials he has seen in getting a new circuit installed at Immokalee--though this relates to CNS Network Systems and not OSG. They started with that back in 2009 when the money was available and it has now been well over a year without results. He didn't mean to lay blame on anyone in particular, but the fact remains that it is just not getting done; Kevin feels that if it had been left to the REC it would have been done in less than a month. The bureaucracy that comes with all this chain of command seems paralyzing to progress.

When Steve joked that it is now "UF & Immokalee" [along the lines of "UF & Shands"] Kevin responded that he had stayed home for this meeting today so that he could guarantee a good connection.

New Microsoft Campus Agreement signed December 16th

We are still awaiting exact details, but Dan Cromer had disseminated the following to the ICC-L:

The formal announcement was apparently delayed and on January 6th Dan sent out another message providing further details:

When Joe Gasper responded asking why OCS was not in the list of licensed CALs for students, Dan responded "I would have hoped to have OCS/Lync as part of the contract, and it may be, but just not mentioned in the announcement I saw."

After some other discussion on the CCC list, Erik Schmidt cleared up many of the details via the following post:

Steve noted his great relief upon reading this posting from Erik because the question of whether or not eCALs would continue for us has now been answered. Andrew cautioned, however, that while we will continue to have eCALs for faculty and staff, students do not. Consequently, students who are not also employees are not licensed for Office Communicator.

Dan Cromer stated that once the Exchange 2010 migration is well under way he has a commitment from Iain that he will let Luis work on Lync. This is one reason we need to get to Windows 2008 R2 native mode for UFAD. Lync will allow federation with outside groups that should permit students to participate in meetings held here via Lync. Student will thus not be able to host meetings, but they certainly can attend them.

Videoconferencing and WAN discussion

[In order to make meeting participation more efficient for Patrick Pettus and James Moore, these two topics have been moved to reside at the top of our agendas.]

Videoconferencing topics (previous discussion)

Connecting to the bridge via Office Communicator

Steve noted that he had received a couple of questions on how to connect to the bridge for the ICC meetings using Office Communicator. The definitive description of that process is available on the UF IT Wiki under the Office Communicator topic within the section labeled "Office Communicator with videoconferencing bridges".

Steve would also like to note that the web streaming instructions he sent out were incorrect because this year the conferences are being held on the second bridge. The ICC Meeting has now been updated to reflect that, but basically, VCs with Conference IDs of 7831xxx are at 128.227.156.82 and VCs with conference IDs of 7832xxx are at 128.227.156.83. He apologizes for any inconvenience.

Polycom status

Francis Ferguson asked how to go about renewing the maintenance on Polycom systems. Dan Cromer responded that this could be handled by contacting Lance Cozart with the serial number of the endpoint. Lance will negotiate with the various vendors to determine the cost.

Office Communicator infrastructure status (previous discussion)

No updates available...

Recording lectures for Distance Education (previous discussion)

Considering AMMS for Accordent management

No updates available...

WAN transition to CNS (previous discussion)

Updates from James Moore

James was unavailable for his usual update this month, but Kevin Hill hoped for a status update on the new circuit which has been pending at Immokalee. Dan Cromer responded that this process has been a real "ping-pong" game between UF Purchasing and Comcast on the wording of the contract. Dan understands the ball is now in Comcast's hands to come up with the final agreement; UF Purchasing is ready and raring to go otherwise. The PO was cut for the PAETEC circuit for Belle Glade, but they didn't have a signed contract. The vendor wanted to put in a penalty clause for early cancellation and Dan had to work with Purchasing to okay that since it initially raised some flags.

New wireless access policy!

Dan Cromer reported that Rob Adams, UF's new security guy, has decided to allow a shared password provide access to wireless. While this is particularly good news for the RECs, it should make certain things easier--especially at remote sites. A password can be generated for wireless access that is changed, perhaps, for each meeting or at some other efficacious period. This would allow prior practices to continue without the need to create individual GatorLink guest accounts for all wireless users. Dan is sure this is welcome news.

Policy

Alternate IFAS domains in e-mail

Steve asked how important this topic was to folks and Andrew Carey responded that it had been important mainly to Dwight from the standpoint of the central Exchange organization. Steve noted that it can also cause confusion for the users. He explained to the more recent folks that IFAS had accounts back in the IFASDOM days and email addresses from those still remain as aliases. A number of people had different usernames under IFASDOM then they do under GatorLink. Users are often confused that one can't universally change an "ifas.ufl.edu" or "mail.ifas.ufl.edu" address to its "ufl.edu" equivalent by just changing the domain portion.

Chris Leopold suggested that we should at least get people started unsubscribing from lists that they might have joined via these other domains. That would be a necessary step in cleaning things up in any case.

Identity Management (IdM) Interface Training

Steve wants to remind everyone of the "UF_PA_IDM_NETMGR" role which will allow you to set NMB for your users. Your Department Security Administrator can do that for you.

Sakai e-Learning System now in production (previous discussion)

One-time GatorLink name changes halted temporarily by Sakai design flaw

Dennis Brown raised the point and Dan Cromer confirmed that Sakai has utilized GatorLink usernames rather than UFIDs as the key it uses across its database. Consequently, individuals wanting to make use of the "one-time" GatorLink name change which has been allowed prior will have to wait as that would prevent them from being able to use Sakai. [See Known Issues]

UF moving to BigBlueButton

Dan Cromer mentioned in passing that UF is looking into replacing Elluminate with BigBlueButton

There has been a hitch in UF's LOA1 plans

Steve asked Dan Cromer if they were still considering allowing the use of Sakai for outside collaboration--at least until the LOA1 implementation was completed. Dan responded that the LOA1 project has run into a snag. It turns out that the UF definition of LOA2 doesn't match well with that of the Internet2 group and that has caused UF to defer development at this time of a means for anyone to generate Gatorlink credentials via the web. Dan never addressed the original question and Steve regrets not following up.

Better access to ongoing discussions?

Steve asked how the general IT community might find out about these sorts of things. Where are they discussed? Are they written down anywhere? Dan responded that there may be some notes from meetings but these things are still basically preliminary discussions. As soon as there gets to be more agreement about the course we are going to take, he assumes details will be published.

Steve wondered why broader input isn't sought regarding what sorts of things our faculty, students, staff want to be able to do. Dan pointed out that small committees get together to figure out basic things with the concept that each member represents some group and is keeping their best interests in mind. It is a distraction to have too many inputs to such processes.

myuf Market (previous discussion)

Steve wants to keep this on our agendas in case discussion seems warranted.

UF Exchange Project updates (previous discussion)

UF Exchange upgrade

No updates available...

Barracuda load issues - replacing with Proofpoint

All ICC members were moved away from the Barracuda to Proofpoint on December 13th. We should each have received a welcome letter from James Oulman:

After several days of not seeing any messages quarantined, Steve began a brief discussion on the ICC-L. James Oulman joined that list and responded to questions:

There seemed to be a bit of confusion on the part of some as to what the user-controllable settings did exactly, but as mentioned in the welcome letter, documentation is now available on-line covering Using Proofpoint and Proofpoint SPAM Settings and Scoring.

Dan sent out an IFAS-wide announcement just as he had proposed:

The actual implementation now appears to be scheduled for one week prior to the end of this month.

Steve Lasley admitted to having not closely looked at the Proofpoint settings levels since he had seen no increase in spam to the inbox and little to nothing in quarantine. Gasper, however, raised the point via the ICC-L that the controls are quite limited compared to what we had experienced with the Barracuda. The four available control settings are also poorly named:

Steve began to wonder why we had significantly more messages quarantined by the Barracuda than are being quarantined by Proofpoint given the following flow:

Proofpoint
Scores

 100
  98  --------\
               \
                \
  80  -----\     \
            \      NOW: Proofpoint Quarantined (default settings)
             \   /
              \ /
               \
              / \
  50 --------/   \
                  \
                    PREVIOUSLY: Barracuda Blocked/Quarantined/Delivered
                  /
                 /
                /
               /
              /
             /
            /
   0 ------/

It does seems odd that the Barracuda would have quarantined several times more messages than Proofpoint does given that Proofpoint had first crack and was discarding the 80-98 range prior but is now sending them to quarantine.

James responded to earlier ICC-L comments publicly:

Steve wants to point out that James has been very open and responsive about this service. Just one indication of this is that he increased the quarantine retention time almost immediately once it was pointed out that the Barracuda settings were longer.

Summary of this week's Exchange Tier 2 and Exchange Policy meetings

Dan Cromer reported that the technical Tier 2 folks have recommended changing the current settings options for the Proofpoint quarantine:

They want to eliminate the "Highest spam protection policy" setting because they feel messages scored between 80 and 98 should always go into the quarantine. For security reasons they also did not want messages scored 50 or above to be delivered to our inboxes; neither do they feel allowing one to disable the Quarantine is a good idea. The default will remain as "Medium spam protection policy" and it is suggested that only two options will be offered: Medium and Low. This was approved by the Policy committee and is being passed onto to Tim Fitzpatrick for consideration.

The wording of those two options unfortunately is not adjustable at this time nor is it clear from this page that "Default means "Medium". It will be investigated whether or not this page can be customized.

Dan would like to propose that an advanced "Custom" setting be considered as well along the lines of the Barracuda that would allow setting the various cutoff scores individually between blocking, quarantine and delivery.

Wayne Hyde noted that blocking includes much more that simply those messages scored > 98 (previously >80). DNS blacklisting and virus protection likely account for the vast majority of total emails blocked.

What surprises Steve is that messages scoring between 80 and 89 were formerly blocked and should now show up in the quarantine; surprisingly there is still very little in quarantine--somewhere in the neighborhood of six times less than with Barracuda for Steve (he has saved all the Barracuda and Proofpoint reports since day 1). It is almost like we are arguing about how to better control "nothing" because there is nothing there to control. Wayne agreed that they might as well just turn off the Quarantine because it really isn't doing anything.

Centralized FAX service via Exchange (previous discussion)

No updates available...

Split DNS solution for UFAD problems

Steve wants to keep this on the agenda for future reference.

IT survey is coming (previous discussion)

No updates available...

Outsourcing of DE course development (previous discussion)

Updates not available...

Projects

New web cluster

More about cleanup of IF-SRV-WEB prior to the move is to come, including that wmv and wma files on the video server need to be moved to media/video server

MPS/DC refresh

No updates available...

New SQL cluster

No updates available...

New virtual infrastructure being planned and spec'ed out

No updates available...

IFAS WebDAV implementation

There continues to be no progress on the documentation which was to happen prior to announcement. Since this has never been formally announced, the matter remains on the agenda as a standing item.

Windows 7 Deployment via the WAIK, MS Deployment Toolkit 2010, USMT 4.0, WDS, and SCCM

Ad hoc UF SCCM Support Group formed

Steve noted that a new UF SCCM list has been created (send a message with the command "SUBSCRIBE SCCM-L Firstname Lastname" in the body to listserv@lists.ufl.edu) and an ad hoc group has developed in order to share information about SCCM. An inaugural SCCM Support Group meeting was held Thursday from 2-4PM in McCarty D Room G001 which was attended by many across campus--including our own Andrew Carey, Kevin Hill and Dennis Brown.

Steve asked Andrew how the meeting went and Andrew responded that he believed the meeting was useful. It was more of a discussion get together than it was a meeting for planning a centralized SCCM structure. IT at the Provost's Office has done considerable work with SCCM as has CLAS and it was helpful to Andrew to hear about some of the things they are doing. Apparently, a good deal of information has now been posted on the UF IT Wiki concerning what they have done and how they've done it.

A centralized SCCM hierarchical structure?

Kevin Hill said he did not know many of the people there but wondered if OSG had any representation. Andrew responded that Erik Schmidt and Mike Kanofsky were there as well as David Burdette from CNS. The reason Kevin asked was that Erik Schmidt had raised the possibility prior of hosting a "root" SCCM server which could be the start of a useful hierarchical structure for SCCM here at UF.

Andrew responded that he believed getting an UF hierarchy started is going to take a bit more grassroots effort on our part. Getting the Provost's Office and CLAS behind that effort might be the best means to that end, but Andrew isn't sure how disruptive that might be to their ongoing work or how amenable they might be to that.

Chris Leopold noted that the SCCM product is still not very mature and wondering if holding off on things for a bit might not be a good idea. Andrew responded that if we begin discussions about a collaborative SCCM structure now, by the time SCCM 2012 is ready to roll UF may then be ready to move.

Exit processes, NMB and permission removal (prior discussion)

No updates available...

Re-enabling the Windows firewall (prior discussion)

No updates available...

Services Documentation: Is a Wiki the way? (prior discussion)

No updates available...

Operations

Restoration of back-ups on the file server

Wayne Hyde intends to document and announce proper usage as time permits.

Membership of ". IFAS-ICC" email distribution group to be narrowed to ICC members only (previous discussion)

Steve will keep this as a standing item on our agendas for now as a reminder. The ICC distribution list is more targeted and restricted to IFAS IT support folks only.

IFAS efforts toward Green IT (previous discussion)

No updates available...

Creating guest GatorLink accounts: singly or in bulk (prior discussion)

Steve had left this on the agenda in case further discussion was deemed warranted.

Can IFAS support DirectAccess in the future? (prior discussion)

Steve wants to keep this topic on our radar.

Moving away from the IFAS VPN service (previous discussion)

Steve assumes that moving our VPN to private IP is waiting on Wayne Hyde finding the time to implement.

VDI desktops as admin workstations (previous discussion)

This is another cool service that Wayne has in progress and which is awaiting sufficient time to pursue further.

Wayne's Power Tools (prior discussion)

No updates available...

Computer compliance tool in production (previous discussion)

Updates not available...

Folder permissioning on the IFAS file server

You are reminded to please take the time to read and implement the new standards. If you have any questions get with Wayne or Steve.

Disabling/deleting computer accounts based on computer password age

This is yet another matter for which finding time for implementation is proving difficult. Steve wants folks to remember that Andrew Carey has a good plan for dealing with this which he simply has had no time to address. In the meantime, it would be very good of each OU Admin to consider mimicking the proposed plan manually by keeping their own records and deleting any computer object which have been disabled for 90 or more days; Wayne's Power Tools can identify those. Steve has finally begun doing that for his own unit and it has made his view within ADUC much more agreeable.

Core Services status (previous discussion)

DPM machine issues now resolved

Chris Leopold reported that there were a number of problems initially with the DPM server that have finally been resolved. Three drives were lost in one array (RAID 6--Uh-oh!) and two drives in the other. They were ping-ponged among three different Dell technicians trying to solve the issues. In solving this, a motherboard was replaced as was a riser. There were several firmware flashes involved and the RAID controller was replaced twice along with driver upgrades. The bottom line is that it is working and luckily no snapshots were lost in the process and nothing had to be recovered from tape.

Steve noted that when the rest of us out in the units are thinking everything is just going smoothly, centrally the IT server administrators are often sweating bullets. Chris agreed that this was not a pleasant process, but the end outcome has been fine. This did affect the enduser's ability to restore previous versions of files for at least one week, but backups were still available should they have been needed.

ePO updates

No updates available...

Status of SharePoint services (prior discussion)

IFAS migrating to centralized MOSS

No updates available...

Public folder file deletion policies and procedures status

Nothing further was available on this topic at this time.

Patching updates...

Microsoft

The January Microsoft patches included only two bulletins. The first bulletin is rated as "Important" and affects Windows Vista while the second bulletin has an aggregate of rating of "Critical" and affects all supported versions of Windows.

McAfee provides podcasts on the highlights of each month's offerings and another podcast summary of these patches is provided by "Security Bulletins for the regular IT guy".

There are two Security Advisories, 2490606 and 2488013, which will not be addressed this month. There now is a "fix-it" workaround available for the latter prior to a true fix.

MS Office News update

No updates available...

Job Matrix Update status

This is here as a standing topic--no discussion this month.

Remedy system status (previous discussion)

No updates available...

Other Topics

UAC settings egregious for users?

No updates available...

PDF-Xchange (prior discussion)

No updates available...

A number of our usual topics were deferred to next month's meeting due to lack of time. The meeting was adjourned on-time at about noon.