When we began our project to implement Banner back in 2000 or so we also began a parallel project to build an EDW. Given the size and scope of the Banner implementation it was decided that the EDW project should be outside the ERP project administration. Thus we ended up with 2 new departments, one to implement Banner and one to implement an EDW right along side of the ERP implementation. Once the Banner and EDW projects were complete the Banner project department was disbanded, however the EDW department continued on, basically only the funding source changed. Many of the technical folks who implemented the Banner system transferred to the department that cares for the administrative enterprise systems. That department contained the DBAs, UNIX admins, developers, etc. That department is where I live today along with my staff of DBAs. BTW, during the project I served as a technical analyst on the Finance implementation team. I'm in my 3rd year as manager of the DBA team, a job that opened up after the project completed.
Each implementation department hired a consulting firm to assist with their project. The consulting firm for the EDW department has subsequently been blamed for the predicament we find ourselves in now due to their design advice. They are gone so it's easy to cast blame on them. I will not disclose any names here or even privately because I have no hard evidence to prove that they were indeed the root cause of our now 14 hour ETL load problem. Most of the people who managed the EDW department at the technical levels are also gone now and other folks have inherited the problems. My team is working closely with the EDW technical folks on solutions to resolve the problem and I'll cover some of that when I get to the current and future discussions but you should know that having 2 separate departments has created it's own unique set of problems of the political/organizational kind.
So this is your first opportunity to learn from what we did. I think the two separate projects allowed each to concentrate on their objectives and to both deliver on time. It was not perfect and some functional folks were needed by both projects at the same time which presented some challenges. The EDW team spent a good deal of time with the campus departmental users to identify data needs. The people on the Banner project teams sometimes felt like they were not given enough input to the EDW design, especially since they were the Banner "experts". However in most cases they would not have had the time to assist even if they were included due to tight timelines on the Banner project. I think this model worked well since the primary audience of the EDW here has been more focused on the departmental users rather than central admin users. If you're building an EDW for executives or central admin folks only then you should probably tightly integrate the EDW and Banner projects.
Once the projects finished the decision to keep the EDW department separate from the department where the Banner support fell is probably the more controversial one and has indeed led to a number of issues over the years. Many believe if the technical staff of the EDW department had been rolled into the same department as the Banner technical staff that we would be better off because we would have the same set of priorities, have more people to throw at problems, etc. As it stands the funding for the EDW department dropped dramatically at the end of the project and that affected staff numbers. Trying to maintain technical staff in that department to work on new EDW development, maintain the current product and respond to minor issues, and to fix developing problems like the 14 hour ETL window have caused lots of thrashing. The EDW department has also been hit with a good deal of turnover and being a small department that makes a bigger impact. If the technical staff had been rolled into our department that supports all the enterprise systems we would have had broader coverage in areas like ETL developers, data modelers, etc. and turnover would make less of an impact. On the flip-side, the priorities/focus of my larger department would also be much broader as compared to the priorities and focus of the EDW department and that may have kept the amount of new data projects from going forward having an impact on what could be delivered to the customer base. I suspect many of you have your own centralized versus decentralized technical staff kinds of issues and this is very similar.
The fact that the DBAs play a big role in the EDW development and maintenance is probably the biggest issue here. The EDW dept. could have also had their own DBAs but because the DBA team has always been in my current department and provided support to both the Banner and EDW project teams it seemed natural to leave them in one place. For the EDW department it is the one resource that is not totally under their control when it comes to their product/environment and it has been problematic over the years. I am fortunate to have 8 DBAs on my team. We bring a broad array of strengths and experience with the 8 DBAs to the table and to the EDW dept. even though they only contract for 2 FTE from us. If the EDW dept. had their own 2 DBAs they would not have the breadth of skills and experience we bring yet they would have their undivided time. Training-wise I think this model works better because the database area is constantly changing and if you are trying to get everything done with only 2 DBAs and also trying to keep them up to speed it is very difficult and expensive too.
If you want some more insight into the two department thing then email me. While there are many problems and many strengths with our model the thing I would recommend most highly if you do go with two groups is to foster a spirit of cooperation and recognize that yes there will be two sides to every issue but those sides are not “right” and “wrong”. It does none of us any good to play those kinds of games although it is an easy trap to fall into. Avoid the “us” versus “them” mentality and you’ll be just fine. An attitude of cooperation has to be fostered by leadership (not just management) in both organizations for it to be successful and it must be practiced and not just preached.
Now that you know how we got started my next installment will deal more with the technical design of the EDW and where it began.
Recent comments
2 days 11 hours ago
1 week 6 days ago
2 weeks 16 hours ago
2 weeks 1 day ago
3 weeks 3 days ago
4 weeks 12 hours ago
5 weeks 5 days ago
5 weeks 6 days ago
6 weeks 1 day ago
6 weeks 1 day ago