Case Study Challenge - Small Agency Legacy Application Migration

By Scott Simpson

All people and situations are fictional. No real people are depicted in the scenarios below. All images are AI generated and do not intentionally mimic any specific person.

Background

A small federal agency has a legacy on-premise application that is twenty plus years old. The application is used to interact with the American public in order to meet the agency’s primary mission. Due to its age and design, the application is difficult to navigate and currently requires a human staffed help desk to walk users through the process. Even with this assistance, the application has a high percentage of non-completion, greater than 75% abandonment. Approximately 1 million citizens attempt to use to the application each year.

The annual operations and maintenance costs are continually growing and as a small agency it is eating up more and more of the available budget. After years of requests, the agency finally received enough funding to begin the modernization process. However, because of a five-month continuing resolution, the funding was only just received and it now must be obligated within seven months, before the end of the fiscal year. Some market research and discovery has been completed in anticipation of receiving funds, but neither is complete.

The agency is looking for a cloud-based application that includes modern business intelligence processes. The new application should also better match the American public to the services offered by this federal agency. It must integrate with the finance system allowing for funds transfers to citizens.

Scenario 1 – Market Research

Personas

Photo	Name - Role	Description
	Reva – Product Owner:	Reva is a long-time agency employee who has worked for multiple offices and is very knowledgeable about all of the agency’s missions. She is an Agile enthusiast, with a strong preference for customer focused design. She is pushing hard for flexible solutions to both the technical and procurement challenges. Reva is very friendly and polite. She is a natural leader who listens to everyone’s ideas and tries to find common ground.
	Lars - PM/COR	Lars has been with the agency for about six months, but has 10+ years of experience with the Department of Defense. He is used to doing things “the DOD way” and is struggling to adapt to the smaller agency lifestyle. He is very risk averse, but open to new ideas that are practical and proven. Lars has a brusque demeanor that many people find off putting. He often interrupts people mid-sentence with information that is off topic or irrelevant.
	Anton - Chief Information Officer	Anton has floated back and forth between Government and industry for 25 years. He joined the agency about 18 months ago, but is still looking to make his mark. Rather than build a coalition of partners to implement his ideas, Anton usually just releases new policies or mandates change. This has not been received well by agency personnel, including CXO colleagues.
	Gabriella - Chief Procurement Officer	Gabriella is a GS-15 under the Chief Financial Officer. She has 20+ years of procurement experience with civilian agencies and zero knowledge of non-FAR authorities. She is extremely customer focused and has a strong desire to meet mission. Last year an internal audit found minimal errors in the Procurement Office, however, the report noted that the Office did not do enough to meet competition requirements.

Situation

Reva (Product Owner), Lars (PM/COR), and Anton (CIO) have been working together for months to create user stories to support the legacy system migration. Anton is meeting with the Gabriella (CPO) for the first time hoping to begin the solicitation preparation.

Speaker	Dialogue
Anton:	Gabby, Congress approved my funds – finally! I'm ready to move out on the on-prem application transition.
Gabriella:	I appreciate all the work you've done in preparation for this; however, I just don't think the package is ready. You've got all these user stories, which I'm unfamiliar with, but I don't see a statement of work or a PWS. Since you said this was an IT contract there is a strong preference to use performance-based contracting. Have your PM check out FAR Part 37.
Anton:	I've got funding that expires in 7 months. I can't go back and start all over again. What am I supposed to do?
Gabriella:	Your office just has to document the market research and get this PR submitted by the end of next week. If it’s received after that then you're going to need to route a waiver for approval up to the Chief Financial Officer, which is way above my pay grade. I’m just a 15 doing my best.

Hearing the news, Reva quickly organizes a call with Lars and Anton to discuss their strategy.

Speaker	Dialogue
Lars:	I say we just recycle the SOW from last time and submit that. It got us this far.
Reva:	That might be the fastest way, but I don’t think it’s the right way. If we just use the last SOW, our users are going to have the same old challenges. We should award a quick 8(a) contract now to perform more discovery and have them create the PWS or whatever it was Gabriella mentioned.
Lars:	Reev, you’ve got no idea what you’re talking about. I’ve been working on this IGCE for months and I’m not going to start from scratch now.
Reva:	No one is even talking about the IGCE, Lars …
Anton:	Okay, let’s take it down a notch. Reva, I don’t think your plan is going to get our funds obligated by the end of the year. We have enough information to make some quick changes to the SOW from last time and get it done. If we don’t get this thing started soon, we’re not going to get my system done in time. By the way I've got a vendor I'd like you to talk with, it's someone I worked with at my last agency. I know we can't direct it to them but I want to make sure that they have a chance to compete for this. So, make it happen.
Lars:	Great, more work to do. Are we ever going to talk about contract type? I’m going to need at least a few weeks to draft the determination and findings that supports a cost reimbursable contract type.

Objective

The program office must:

1. perform Market Research, and
2. draft a requirements document.

You work with the Program Management Office (PMO) as a liaison between the program team and the procurement office (sort of like a procurement analyst). Provide advice for how the program office could perform their two objectives, identifying potential challenges they may experience and possible solutions.

Review with Reva and Lars common agile principles that should be (or should have been) considered when developing a public facing application.

Potential Solutions

Have an in-depth conversation with Reva and Lars together

Speaker	Dialogue
PMO Liaison:	Lars, I understand that you’re worried about the IGCE and a D&F for cost reimbursable contract type. Let’s put that off to the side for now and talk about it again later. It’s not that it’s not important, but we can make the decision on contract type after we perform some market research. Maybe it’s even a question that we might want to put into the RFI. What do you think about that?

This puts the concern into a “parking lot” for later.

Speaker	Dialogue
Lars:	Alright, whatever. If this thing isn’t awarded on time, it’s not my fault.
PMO Liaison:	Let’s post a quick request for information to SAM and tell vendors that this is most likely going to be awarded this fiscal year.

What type of information do you ask for? Do you send a copy to the CIO’s vendor?

Speaker	Dialogue
PMO Liaison:	After we look through the RFI responses, we can hold one-on-ones with some of the vendors to ask follow-up questions in real time.
Reva:	Is that allowed?
PMO Liaison:	Oh yeah, it’s fine. OFPP, that is, the Office of Federal Procurement Policy, released this whole Myth Buster series on industry communication years ago. I’ll send you a copy.
Lars:	Sounds like a waste of time. What would we even ask them?
PMO Liaison:	Well, I put together some draft questions based on what we’ve talked about, but we can ask them other things too.

Potential Questions to ask:

• Whether discovery should be done before award or if it can be included as part of the overall contract.
• What the best type of statement of requirement is for this type of work (SOO, SOW, or PWS).
• Ask about potential evaluation methods for the solicitation phase.
• Ask about possible technology solutions to better match the public to services (including AI, etc.)

Are there any others?

Speaker	Dialogue
Lars:	These are good questions. Do we have to talk with everyone?
PMO Liaison:	Nope, just the ones you want to. Maybe five RFI vendors?
Reva:	What about discovery? We’re not done yet.
Lars:	What is your hangup with discovery?!
PMO Liaison:	Discovery is actually something we do every day, we just don’t call it discovery. If you’ve ever use Netflix or Pandora, you’ve been part of a discovery session. Those two apps ask what you like and don’t like and then recommend new content based on your answers.

What other every-day examples could you use to describe discovery?

Speaker	Dialogue
Lars:	Oh my goodness. Have you watched Black Mirror?! It is so good!
PMO Liaison:	I’ll have to check it out. But you know, Reva. Discovery doesn’t have to be done before you start. Most agile companies can incorporate discovery sessions into their sprint plans, it’s just dedicating time to user feedback and iterative experimentation. We should ask some of them during our one-on-ones.
Reva:	Okay, but that doesn’t solve our problem about the SOW or PWS. I need to get that written in like 2 days or I have to go through this whole PR waiver process.
PMO Liaison:	What if we used a statement of objectives? That’s a lot easier to put together and it actual works better for an agile procurement anyways.

Tell them more about a SOO.

Speaker	Dialogue
PMO Liaison:	In fact, if you want, we can write the SOO together. It will only take a couple of hours, tops. And then you’re done!
Reva:	Okay, you’ve convinced me. Let’s go for it!
Lars:	Where did you learn about all of this stuff?

Additional Questions

• What other information would you share with them?
• How might Anton (CIO) have prevented some of this from happening?
• What else could Gabrielle (CPO) do to help ensure timely award?

Agile Overview

Speaker	Dialogue
PMO Liaison:	Before writing your requirements, you should spend some time gathering user stories from all of your stakeholder offices.

Tell them more about user stories.

Speaker

Dialogue

PMO Liaison:

You should also consider holding discovery sessions with public users, especially those ones that didn’t complete the interaction with application. It would be great to know why they exited the application. Another great best practice is to design with the end in mind. This would mean talking with offices like security, privacy, accessibility to ensure that all of their requirements (ATO, 508 compliance, US web design standards, etc.) were included from the beginning. It would also help to avoid any surprises later in the process.

What other agile best practices you could advise the team to consider regarding their overall migration?

Scenario 2 – Solicitation

The Market Research Report and statement of requirements was submitted on time and the funding was obligated to the PR. Gabriella (CPO) has assigned a Contracting Officer and Attorney-Advisor. When Gabriella read over the statement of objectives, she had concerns about gathering citizen data and shared the document with the Chief Privacy Officer, who is now sending a representative to attend IPT meetings.

The initial CIO has left the agency to join a venture capital firm and a new CIO was quickly appointed from industry.

Photo	Name - Role	Description
	Helena - CO:	Helena joined the agency last year but only just transferred to the office supporting IT requirements. She held a $5 million warrant at a previous agency, but procured mostly professional, non-IT services from GSA schedule. Helena feels a little unprepared since the team has been meeting for several months without her and is struggling to fit in with the others. As the CO, Helena is very unsure of herself and often relies on others to make decisions or tell her whether something is allowable.
	Maya - Attorney-Advisor	Maya (they/them) tries to attend most meetings, but is often not present. When asked about whether something is required or possible, they often demur until they see the question/document in writing. Upon providing legal advice, Maya rarely cites statutes or GAO decisions and instead makes vague statements about the legality or risk of something. Maya uses very complex language and often drops legal jargon and phrases that need explanation, taking up precious time during the meetings.
	Kim - NEW Chief Information Officer	Kim has joined the agency from industry and is looking to make her mark. She has the grand idea to merge several systems together into one larger one to operate as a central place of record. Kim has already discussed this with other CXOs and mission leaders, who are excited about upgrading their systems and want a lot of new functionality.
	Nate - Privacy Officer	Nate has worked in privacy about seven years. He started off in IT, but has a passion for data privacy, especially as it relates to emerging technology.

Situation

Reva, the Product Owner, is meeting with Helena, the CO, for the first time. Helena has forwarded the invitation to Maya, attorney-advisor, and Nate, privacy officer. Lars is out of the office on vacation.

Speaker	Dialogue
Helena:	I’m the Contracting Officer that’s been assigned to your PR, the one about building some kind of legacy fund.
Reva:	What? No, it’s about modernizing a legacy application that is currently housed on-prem and migrating it to a cloud-based environment, preferably FedRAMP aisle 3 at least.
Helena:	Oh, sorry. Maybe I got mixed up. So anyways. The package included draft evaluation factors, but that’s not normally something the program office handles. I got a sample from my Branch Chief and I’m going to use it instead. Is that alright?
Reva:	Well, I’d like to see the final before it’s released. I really think that our technical evaluators need to see the offerors perform some coding and ask them questions about their work so that they can pick the right vendor.
Maya:	Let me just jump in here and say absolutely not. According to FAR 15.306, it is not permitted to have discussions with offerors until a competitive range is established. If you’re going to ask questions of the vendors then we’re going to have to give them a complete list of all of their strengths and weaknesses. I don’t think I’m prepared to approve that.
Helena:	Thanks, Reva. Okay, so asking questions are off the table. What about this – coding challenge thing?
Maya:	I don’t think it’s a good idea. A few years ago, I had another team try to evaluate a coding challenge. However, the Git repository wasn’t setup correctly and none of the code was saved. The agency had to request that all offerors redo their entire coding challenge. That team was extremely lucky that they didn’t receive a protest, a protest which I firmly believe the agency would have lost. We might not be so lucky this time and there is zero case law out there right now about coding challenges.
Reva:	Anyone else got any bad news they want to share?
Nate:	I’m actually really curious to know whether any of your potential solutions will consider artificial intelligence?
Reva:	Yeah, they might. A few of the vendors we talked with as part of our market research had new modules coming out that incorporated the latest in AI.
Nate:	Not to be the bearer of bad news, but that’s going to pose a challenge. I’ve heard through the grapevine that there’s a new Executive Order coming out of the White House about AI.
Reva:	Another one?!
Nate:	Yeah, and it applies specifically to applications that interact with the American public. I don’t have a copy of the language yet, but it could be a game changer. The Chief Privacy Officer is suggesting a pause on all AI procurements until the EO is released so that we can ensure we’re in compliance.
Reva:	How long is that going to take?
Nate:	I dunno.

Kim, the new CIO, scheduled a meeting with Gabriella to find out the status of the legacy application transition.

Speaker	Dialogue
Kim:	Gabriella, thanks so much for meeting me. I know that it’s third quarter and you’re probably swamped, so I’ll get right to business. I’m worried that if our requirement for the legacy app transition isn’t awarded soon, the Administrator is going to sweep my funding and give it to mission folks, which is going to be bad for us. We’ve only got five months left in the year!
Gabriella:	My team has looked over the PR that was submitted and it looks like they have everything they need. The Contracting Officer should be working it now and be able to award it before the end of the year. Have they sent you the milestone schedule yet?
Kim:	That’s just it, when they talked earlier this week it was not good news. What I’m really wondering is if we can turn this into a sole-source to get it awarded as fast as possible.
Gabriella:	I don’t think we’re quite there yet. Our competition numbers aren’t doing real well this year, I’ve already signed more LSJs than I’m comfortable with. How about this. I have an Acquisition Innovation Advocate on my team, they usually have a lot of great ideas. I’ll have them meet with you and your team as soon as possible to make sure this is on track to be awarded by end of FY.

Objective

The CPO has requested innovation coaching to support this procurement and ensure it is awarded on time. You are acting as the Acquisition Innovation Advocate and innovation coach.

Advise the team about an acquisition strategy that will be approved by all stakeholders.

Potential Solutions

As the AIA, you setup a meeting with Helena (CO):

Speaker

Dialogue

AIA:

This all has to be really confusing for you since you’re just joining the team. We should do some ice breakers at our next few meetings so that you get to know everyone! Let’s talk first about your role as the CO. FAR 1.102-5 says that as the CO, you should be the one to take the lead in encouraging business process innovations and ensuring business decisions are sound. Maya, our attorney, and Nate with Privacy, are just advisors to you. I’ve got a few resources that I think might help you streamline this procurement and get it awarded on time. The first one is the DHS Procurement Innovation Lab. They’ve got some trainings you can take and some literature you can read. You should also check out the Periodic Table of Acquisition Innovations, which most of us just call the PTAI. It’s got samples of how other agencies have used innovations and GAO cases that support them. Since you’re new to IT procurement, I don’t know if you’ve heard about the DITAP program yet.

Explain the program.

Speaker	Dialogue
AIA:	Our agency might be small, but we’ve got over a dozen COs that have been through the training. I’ll introduce you to a few and maybe they can help.
Helena:	I heard that the CIO really wants me to sole source the award just to get it done.
AIA:	Five months should be lots of time to award this contract. But, just in case, I’ll reach out to our industry liaison and talk to them about whether they know of a Small Business Innovation Research (SBIR) vendor we can make a quick award to. An SBIR Phase 3 isn’t considered a sole source, the competition was already done as part of Phase 1 and 2, so that should be okay with our CPO. But I’ll talk to her about it too. By the way, That coding challenge Reva suggested is a good idea. It might shave off a week or more from the evaluation time.

Tell her more about why.

Speaker	Dialogue
Helena:	Maya is never going to let me do a coding challenge. And even if she did, I’ve no idea how to run one!
AIA:	First off, Maya’s is only offering advice. There are a few GAO cases about coding challenges. I’ll send them over for you to take a look. Second, maybe you don’t even need a coding challenge. What if you just had an oral presentation and asked the offerors how they would go about coding the application? I don’t think Reva and Lars have the resources to prepare for a coding challenge right now anyways. This would save everyone time, including the offerors.
Helena:	I don’t think that’s going to meet the evaluators’ needs. Maya still won’t let them ask any questions.
AIA:	Well, I’m not sure Maya’s right about that either. The GAO has consistently said that a critical part of discussions is allowing the offerors an opportunity to revise their earlier proposal. Plus, you might not even be in FAR Part 15.3. Maybe you’ll use the GSA Schedule to award this under 8.4 and you can use select best suited, then negotiate instead of discussions.

Tell them more about this.

Speaker	Dialogue
AIA:	And hey, if you still can’t convince Maya, then maybe you can use some static questions, ones that you’d ask every vendor. “Tell me about how you would use the agile principles to migrate a legacy on-prem app to the cloud?” It’ll give the evaluators some great insight. And you can ask them on the spot so offeror’s don’t have a chance to rehearse them.
Helena:	I like that idea!
AIA:	You could invite Maya to attend the oral presentations too. That way they see what’s going on.
Helena:	Ha! Like Maya has time for that.
AIA:	And remember too, if there’s a vendor out there that doesn’t agree with your approach, whether it’s a coding challenge or an oral presentation, they have to protest the solicitation, before proposals are submitted.
Helena:	That’s a really good point. I’m feeling a lot better about this whole coding challenge thing. What about Nate’s concern with that new AI EO that’s coming out.
AIA:	I know someone from the AIA Council who works over at the Office of Federal Procurement Policy. I’ll reach out and see what they know. But I wouldn’t worry about an EO that might or might not come down before you make an award. It might be another opportunity to use the select best suited, then negotiate approach. Before you go, let’s setup our next meeting so that it doesn’t get lost in the shuffle. How about meeting every two weeks for a quick check in? I usually meet with teams for just 15 minutes and we talk about accomplishments for the last two weeks, goals for the next two weeks, and if there are any challenges.

At the next meeting between Helena and the AIA:

Speaker	Dialogue
Helena:	I’ve been doing a ton of reading since we talked last week. Have you watched these PILCast videos on YouTube? They’re so great! Anyways, I wanted to run my acquisition strategy by you real quick before I route it for review.
AIA:	Go for it!
Helena:	So I’m thinking about a two-phased advisory down select. For Factor 1, Reva has some good corporate experience questions, like ability to demonstrate previous experience transitioning on-prem legacy application of similar size and conditions from to cloud. I think that I’ll advise about three offerors to proceed. If we get four, that’s okay too, but I don’t think we will. I’ll give them about 2 weeks to prepare for Phase 2, which is going to be just an oral presentation and price.
AIA:	Are you going to give them the oral presentation topic beforehand?
Helena:	No, they’re all going to be provided on the spot that morning with just a little bit of prep time. And I took your advice about select best suited, then negotiate, since I’m not in FAR Part 15. I’m going to include that for both technical, price, and clauses. So, if that EO comes down, which thanks to your friend at OFPP is unlikely, we can always talk with the vendor to include it. I’ll use the best value tradeoff to make my award. It should be done 2 months from now. Time to spare!
AIA:	That sounds like a great approach! Great job, Helena!
Helena:	I couldn’t have done it without you.

Additional Questions

• How would you address the new CIO’s idea about including additional systems within the migration?
• Is there any other advice you would give to the Contracting Officer or CPO?
• Is there any additional advice you would give the Program Office?

Learning Outcomes

Learning objectives include applying the agile process, demonstrating knowledge of user centered design, sprints/backlogs and innovative procurement techniques, and recognizing the need for early engagement with stakeholders.

The case study highlights the importance of communication among the stakeholders and users in order to develop an application that responds to the needs of all. Providing accessible digital services to the American public is a crucial challenge that the government is facing.

Specific measurable outcomes include improved efficiency and stakeholder satisfaction, cost savings, and the ability to award a procurement on time.

Instructor Guidance

There are many different ways to present this case study.

• Have different students read/act out roles (PO, PM, CO, CS, etc.) and then identify challenges/blockers and potential solutions.
• Have students read silently, then discuss in groups.
• Have students read just one section aloud, and then respond to just that one challenge.
• Have students read at home and prepare a “skit or scene” for how their challenge might be overcome.

There are also different ways to focus on the overall challenges.

• Technical challenges.
• Agile challenges.
• Procurement challenges.
• Personal interaction challenges.
• Communication/collaboration challenges.