Department of Veterans (VA) Affairs Mobile Applications Cloud Migration Case Study
By VA’s Technology Acquisition Center (TAC)
POC
Juan Quinones, Contracting Officer, Juan.Quinones@va.gov
Jessica Adamitis, Contract Specialist, Jessica.Adamitis@va.gov
Mary Courtney, OGC, Mary.Courtney@va.gov
Contract Information
Title: VA Mobile Applications Cloud Migration (MACM)
Applicable FAR Part(s): FAR Part 12.6 and FAR Part 15.3
Total Task Order Value (Firm Fixed Price): $44.85M
Period of Performance: 12-month Base period followed by four 12-month option periods.
Scope: The Contractor shall implement, operate, and maintain the VA mobile application platform including all cloud services required to support VA MACM operations. The Contractor shall migrate mobile apps and supporting services into an operational VA mobile application platform. The Contractor shall provide O&M support of all VA MACM cloud environments as well as the infrastructure software contained within each environment. The Contractor shall provide all software and support required to create and maintain an operational platform as described in the PWS. This effort also includes an optional task for transition out services.
The Problem
Source selections centered around a traditional written proposal do not allow evaluators to fully evaluate an offeror’s capabilities and/or processes. This is especially true for digital service procurements that are to be developed using agile methods. It is one thing to be a great proposal writer but quite another to be capable of successfully performing. Thus, we needed to come up with an evaluation strategy that required offerors to demonstrate what they would have otherwise just written about. By utilizing a demonstration to supplement a written technical approach, we could make the offerors show us that they could do the work versus just telling us, increasing our confidence in their ability to successfully perform. Information and Technology (OIT), Veterans Health Administration (VHA), and United States Digital Service (USDS) subject matter experts (SME) recognized a need for the offerors to demonstrate their expertise with a practical test of their capabilities. Accordingly, the team (comprised of TAC, OIT, VHA, USDS, and Office of General Council (OGC)) worked to implement a demonstration of the offerors’ expertise into the procurement. The team also recognized that a demonstration could be useful as a mechanism to down select offerors into a more competitive pool for further evaluation, allowing the Government to better focus its resources.
The Solution
The team decided that a Technical Demonstration titled, “VA Mobile Application Cloud Migration (MACM) Minimum Viable Product (MVP) Prototype Demonstration” would be used to determine an offeror’s ability to establish a MACM MVP prototype running a High Availability Kubernetes Cluster (K8S). The MACM MVP prototype demonstration was used as a gate before evaluating the remainder of the submitted proposal which consisted of a written technical response, Past Performance, Price, and Veterans Involvement. After submission of the written proposals, all Offerors were given five business days from the date instructions were provided to prepare the MACM MVP Prototype Demo. The MACM MVP Prototype Demonstration would be evaluated on a Pass/Fail basis. When completed the Government should be able to execute the code and standup the MACM MVP Prototype, and build, test, and deploy the application.
Basis of Award
This competition was accomplished in two steps.
In Step One, the Offeror completed the MACM MVP Prototype Demonstration which was evaluated by the Government on a Pass or Fail basis. Only those proposals determined to Pass under the MACM MVP Prototype Demo Factor were eligible to participate in Step Two. There was no Government interaction or discussions in Step One.
In Step Two, the Offerors were subject to a tradeoff evaluation between the remaining four Factors (Technical, Past Performance, Price and Veterans Involvement) in order to determine which proposals represent the best value to the Government. The technical proposal consisted of a 30-page written technical response to seven technical elements (Appendix pages 16-17) that were evaluated independently from the MACM MVP Prototype Demo Factor in Step One.
Details regarding the Basis for Award, Evaluation Approach and Proposal Submission Instructions can be found in the Appendix, pages 13-22.
MACM MVP Prototype Demonstration
USDS developed an obviously fake crypto coin mining application written in Python, JavaScript, and Ruby that runs as five separate services on a single server orchestrated by Docker Compose with an nginx web server in front. Each service has its own Dockerfile. The api service calls the rng_service and the hash_service to produce hashes. Results are stored in the redis service. The web_front end is the view into the hashing and mining of coins.
The following skillsets were utilized to establish the MACM MVP Prototype Demonstration, and participated in the evaluation:
OIT Program Managers (Requiring Activity)
VHA Program Analysts (Business Owner)
Digital Service Experts (SMEs)
OGC Attorney
Contracting Officer
Contracting Specialist
TAC Program Manager, Engineering Service
MACM MVP Prototype Demonstration Ground Rules and Instructions
- The Offeror shall have five business days to prepare and deliver the MACM MVP Prototype Demo.
- The Offeror shall deliver a code repository to GitHub that is accessible by Government staff for up to two weeks after the MACM MVP Prototype Demonstration for Government evaluation. No additions or updates can be made to the code or any artifacts within the repository after submission. The Government will provide the names, contact information, and GitHub username for access to the MACM MVP Prototype Demo.
- The Offeror will be graded as pass/fail on all items detailed below. To Pass an Offeror is required to Pass all nine items below. In the event the Government can execute code and standup the MACM MVP Prototype Demo, build, test, and deploy the application and service prototypes from a code repository, but the MACM MVP Prototype fails one of the nine items below then the MACM MVP Prototype Demo will be deemed to have failed.
- There will be no extensions or rescheduling of an Offeror’s time to prepare and deliver the VA MACM MVP Prototype.
The Offeror shall establish a MACM MVP prototype running K8S. Any base Amazon Machine Images (AMI) shall be defined via code. The Offeror shall use the following open source or comparable tools:
- Amazon Web Services Command Line Interface
- Terraform by HashiCorp
- Packer by HashiCorp
- Ansible
- Kubernetes
For the VA MACM MVP Prototype Demonstration, the Offeror shall complete the following:
Item 1: Create two Virtual Private Clouds (VPC) inside the same AWS commercial account to demonstrate the VPC to VPC configuration.
- The VPCs shall exist in the same account where VPC1 has direct access to the internet and VPC2 does not.
- VPC1 and VPC2 shall have a connection.
- VPC2 shall not be directly connected to the internet.
- Demonstrate routing from within VPC2 to https://vets.gov
- curl --trace-ascii https://vets.gov
Item 2: Establish a Linux Jumpbox. Item 3: Provision Roles and Users with access to the Linux Jumpbox and instances via Identity and Access Management (IAM) Item 4: Only traffic on ports 22, 443, and 444 shall be allowed via an entry point to VPC1. Item 5: Secure Shell (SSH) ingress traffic is only allowed to the Jumpbox via port 22 Item 6: All configuration must be done via code, nothing shall be updated via the console. Item 7: Deploy a Kubernetes cluster Item 8: Deploy a Government provided sample Web Application to the cluster
- Migrate the provided web application which uses Docker Compose and resides in the following repository to the prototype K8S cluster:
- https://github.com/department-of-veterans-affairs/map-demo
- Decompose the provided application and services as appropriate to be consistent with modern micro-services architecture.
- Setup the web application for high availability
- Access the web application in the K8S cluster via the HTTPS entry point toVPC1
Item 9: The Offeror shall present the VA MACM MVP Prototype Demo to the Government via its GitHub submission.
In this case all three vendors passed the MACM MVP Prototype Demonstration and proceeded to Step Two of the evaluation.
Evaluation Criteria
When completed the Government shall be able to execute the code and standup the MACM MVP Prototype Demo (inclusive of VPCs and all tools used for automation and orchestration), build, test, and deploy the application and service prototypes from a code repository. The Offeror is required to receive a rating of Pass for all items 1 through 9 to receive a final Pass. There will be no discussions as part of Step One. The Government evaluators reviewed the demo submission and objectively determined whether the VA MACM MVP Prototype developed by the Offerors met each of the nine criteria outlined above.
Bottom Line
The goal of the MACM MVP Prototype Demonstration was to determine if the offerors could deploy/migrate a Government provided sample web application to a high availability K8S cluster in a fixed timebox of five business days. Having the offerors prove this out via a tangible demonstration increased the Governments confidence in the offerors’ written technical approach and the capabilities to perform critical tasks. It also offered an opportunity for a potential down select if need be. Note: In this case it was not needed as there were only three offerors whom all passed. It should be noted that including the Technical Demonstration likely contributed to only receiving three proposals.
Lessons Learned
- Some MACM MVP Prototype Demonstration submissions were better than others. As such some evaluators, while appreciating the simplicity of pass/fail, expressed the desire to be able to assign strengths and weaknesses and a rating for the demo to differentiate the submissions.
- The rigidness of having to meet all 9 criteria in a pass/fail scenario may lead to a vendor with an overall great demo to be excluded based on a minor technicality.
- During the debriefs, the offerors provided positive feedback regarding use of the demonstrations and stated they should be used more often.
- A demo of this nature requires SMEs capable to create the test (e,g. fake crypto coin mining application) and capable of evaluating the outcome.
Additional Details
Solicitation Number: 36C10B18R2609
Milestones: Solicitation Issued: August 3, 2017 Proposals Received: August 22, 2017 Demos Complete: August 29, 2017 Evaluations Complete: September 20, 2017 Award: September 26, 2017
APPENDIX
Section B – Price Schedule (Base Period Only):
Please note this effort included unique pricing terms specifically related to the VA MACM Minimum Viable Product (MVP) at CLIN 0002 and with regard to Service Level Agreements. The Contractor shall meet all Deliverable Metrics/Service Level Agreements (SLA) as described in the Performance Work Statement (PWS). However, if the Contractor’s performance falls below a required service level, the Contractor shall only be paid for the lower service level provided. See Section B.5 Performance Work Statement, Section 6.4.1 for Deliverable Metrics/SLA Performance for the calculation and measurement of each Firm-Fixed-Price (FFP) Deliverable Metric/SLA specified.
BASE PERIOD The period of performance shall be for 12 months from award.
| CLIN | Description | Qty | Unit | Unit Price | Extended Price |
|---|---|---|---|---|---|
| 0001 | Project Management in accordance with (IAW) Performance Work Statement (PWS) Section 5.1 and its subtasks. Period of Performance (PoP) shall be for 12 months from award. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.1 and its subtasks. | 12 | Month (MO) | $ | $ |
| 0001AA | Contractor Project Management Plan IAW PWS Section 5.1.1. Due 30 days after contract award (DAC) and updated monthly thereafter. | 12 | MO | NSP | NSP |
| 0001AB | Monthly Status Report IAW PWS Section 5.1.2. Due the 5th day of each month throughout the PoP. | 12 | MO | NSP | NSP |
| 0001AC | TMS Training Certificated of Completion for VA Privacy and Information Security Awareness Training IAW PWS Section 5.1.6. Due 1 week after award and/or one week after on-boarding of new employee and annually as required throughout the period of performance for all Contractor employees. | 1 | LO | NSP | NSP |
| 0001AD | VA Privacy and Information Security Awareness and Rules of Behavior Training Certificate IAW PWS Section 5.1.6. Due 1 week after award and/or one week after on-boarding of new employee and annually as required throughout the period of performance for all Contractor employees. | 1 | LO | NSP | NSP |
| 0001AE | VA HIPAA Certificate of Completion IAW PWS Section 5.1.6. Due 1 week after award and/or one week after on-boarding of new employee and annually as required throughout the period of performance for all Contractor employees. | 1 | LO | NSP | NSP |
| 0002 | VA MACM Platform Development and App Migration IAW PWS Section 5.2. The Contractor shall price 13 two-week sprints in the base contract period. The Contractor may invoice at the end of each sprint upon Government acceptance of the Sprint Certification Package and Artifacts. The VA MACM Minimum Viable Product (MVP) IAW PWS Section 5.2.1 shall be delivered 90 DAC. Mobile Application Migration IAW PWS Section 5.2.2 shall begin upon Government acceptance of the VA MACM MVP IAW PWS Section 5.2.1. The price for the delivery of the minimum viable product (MVP) as defined in PWS 5.2.1 is 100 percent of the fixed-price of each sprint provided the Contractor successfully delivers and the Government accepts the MVP within three months from contract award. Upon Contractor delivery and Government acceptance of each sprint, the Government shall pay 75 percent of the fixed-price of the sprint. Upon the successful delivery and acceptance of the MVP within three months from contract award, the Government shall pay the Contractor the remaining 25 percent of the fixed-price of each sprint withheld at the time of each Sprint delivery. In the event the Contractor delivers the MVP later than three months from contract award, the price for the delivery of the MVP shall be 75 percent of the fixed-price for each sprint. The Government will continue to pay 75 percent of the fixed-price of each sprint until successful delivery and acceptance by the Government of the MVP. Notwithstanding when the MVP is delivered and accepted by the Government, the Contractor remains obligated to deliver the MVP and failure to deliver the MVP within a commercially reasonable time from contract award will be deemed a condition endangering contract performance and may provide grounds for default termination. Upon successful delivery and acceptance of the MVP, additional sprints for the Mobile Application Migration IAW PWS Section 5.2.2 will be paid at 100 percent of the fixed-price of each sprint. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.2 and its subtasks. | 13 | EA | $ | $ |
| 0002AA | Sprint Plan IAW PWS Section 5.2. Due at the beginning of each two-week sprint and updated at conclusion of each two-week sprint. | 13 | EA | NSP | NSP |
| 0002AB | Sprint Certification Package with Source Code IAW PWS Section 5.2. Due at conclusion of each two-week sprint. | 13 | EA | NSP | NSP |
| 0003 | VA MACM Operations and System Maintenance Support IAW PWS Section 5.3.1. PoP shall begin upon Government acceptance of the VA MACM MVP IAW PWS Section 5.2.1. Not to Exceed (NTE) 12 months. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.3.1 and its subtasks. | NTE 12 | MO | $ | $ |
| 0003AA | VA MACM O&M Support Plan IAW PWS Section 5.3.1. Due 30 DAC and updated thereafter. | 1 | LO | NSP | NSP |
| 0003AB | SLA Monitoring Plan IAW PWS Section 5.3.1. Due 30 DAC and updated monthly thereafter. | 1 | LO | NSP | NSP |
| 0003AC | VA MACM SLA Report IAW PWS Section 5.3.1. Due 30 DAC and updated monthly thereafter. | 1 | LO | NSP | NSP |
| 0004 | Security IAW PWS Section 5.3.2. PoP shall be for 12 months from date of award. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.3.2 and its subtasks, except for 5.3.2.2. | 12 | MO | $ | $ |
| 0004AA | Monthly Security Assessment Report IAW PWS 5.3.2.1. Due monthly throughout the PoP. NTE 12 months. | 12 | MO | NSP | NSP |
| 0004AB | ATO Package Documentation IAW PWS 5.3.2.2. Due 30 DAC and updated as needed. | 1 | LO | $ | $ |
| 0005 | Disaster Recovery (DR) and Continuity of Operations IAW PWS 5.3.3. PoP shall be 12 months from date of award. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.3.3 and its subtasks. | 12 | MO | $ | $ |
| 0005AA | DR and Backup Plan IAW PWS 5.3.3. Due 45 DAC and updated as needed throughout the PoP. | 1 | LO | NSP | NSP |
| 0005AB | DR Test Schedule IAW PWS 5.3.3. Due 45 DAC and updated as needed throughout the PoP. | 1 | LO | NSP | NSP |
| 0005AC | AAR IAW PWS 5.3.3. Due 45 DAC and updated as needed throughout the PoP. | 1 | LO | NSP | NSP |
| 0005AD | COOP Plan IAW PWS 5.3.3. Due 45 DAC and updated as needed throughout the PoP. | 1 | LO | NSP | NSP |
| 0006 | Monitoring and Alerting IAW PWS 5.3.4. PoP shall be 12 months from date of award. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.3.4 and its subtasks. | 12 | MO | $ | $ |
| 0006AA | Quarterly Resource Utilization Optimization Plan IAW PWS 5.3.4. Due quarterly throughout the PoP. | 1 | LO | NSP | NSP |
| 0006AD | Monthly Utilization Report IAW PWS 5.3.4. Due monthly throughout the PoP. | 1 | LO | NSP | NSP |
| 0007 | Software License Management IAW PWS 5.3.5. Time-and-Materials (T&M) This is a T&M Contract Line Item Number (CLIN) and includes all labor and material required for the successful completion of the tasks PWS 5.3.5. Work shall be executed in accordance with funding provided at time of award or under formal modification. Work shall not commence unless and until funding is provided by the Contracting Officer. This T&M CLIN is IAW FAR 52.232-7 for invoicing or billing purposes. T&M rates shall not exceed those set forth in Attachment 006. *Government Not To Exceed Material Ceiling: $_____ + Fixed Handling Rate ______ % = Material NTE Ceiling $ | 1 | LO | NTE $ | NTE $ |
| 0008 | Engineering Technical Support IAW PWS 5.3.6. PoP shall be 12 months from date of award. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.3.6. | 12 | MO | $ | $ |
| 0009 | General Engineering Support IAW PWS 5.3.7. PoP shall be 12 months from date of award. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.3.7. | 12 | MO | $ | $ |
| 0010 | Release and Development Management IAW PWS 5.4. PoP shall be 12 months from date of award. This CLIN includes all tasks, labor, and travel required for the successful completion of the services details in PWS Section 5.4 and its subtasks. | 12 | MO | $ | $ |
| 0010AA | Meeting Minutes IAW PWS 5.4.2 Due within 24 hours of scheduled meetings. | 1 | LO | NSP | NSP |
| 0010AB | ECCB SOP IAW PWS 5.4.3. Due 30 DAC and updated monthly thereafter. | 1 | LO | NSP | NSP |
| TOTAL BASE PERIOD | $ |
BASE PERIOD – OPTIONAL TASK – ADDITIONAL VA MACM PLATFORM DEVELOPMENT AND APP MIGRATION SUPPORT This optional task may be exercised IAW FAR 52.217-7 Option for Increase Quantity – Separately Priced Line Item (MAR 1989). Work shall not commence until, and unless, a formal modification is issued by the Contracting Officer.
| CLIN | Description | Qty | Unit | Unit Price | Extended Price |
|---|---|---|---|---|---|
| 0011 | Additional VA MACM Platform Development and App Migration Support IAW PWS 5.2.3. The Contractor shall price per two-week sprints in the base contract period. The Contractor shall begin within two weeks of option exercise. This Optional Task may be exercised multiple times from time to time during the base period, not to exceed 32 sprints. | NTE 32 | EA | $ | NTE $ |
| 0011AA | Sprint Plan IAW PWS Section 5.2. Due at the beginning of each two-week sprint and updated at conclusion of each two-week sprint. | 32 | EA | NSP | NSP |
| 0011AB | Sprint Certification Package IAW PWS Section 5.2. Due at conclusion of each two-week sprint. | 32 | EA | NSP | NSP |
| BASE PERIOD – OPTIONAL TASK – ADDITIONAL VA MACM PLATFORM DEVELOPMENT AND APP MIGRATION SUPPORT | $ |
BASE PERIOD – OPTIONAL TASK – TIER 4 HELP DESK SUPPORT This optional task may be exercised IAW FAR 52.217-7 Option for Increase Quantity – Separately Priced Line Item (MAR 1989). Work shall not commence until, and unless, a formal modification is issued by the Contracting Officer.
| CLIN | Description | Qty | Unit | Unit Price | Extended Price |
|---|---|---|---|---|---|
| 0012 | Tier 4 Help Desk Support IAW PWS 5.5. This is a Labor Hour Contract Line Item Number (CLIN) and includes all labor required for the successful completion of the tasks PWS 5.5. The Contractor shall not execute performance on this task without approval from the Contracting Officer via a modification. The Offeror’s fully loaded labor rates shall not be exceeded and the rates are incorporated into the contract at Attachment 006. The binding fully loaded labor rates will be used for the prime and all subcontractor/team members in performance of this contract. This Labor Hour CLIN is in IAW FAR 52.232-7 for invoicing or billing purposes This optional task may be exercised multiple times from time to time up to the NTE amount. | 1 | LO | NTE $ | NTE $ |
| BASE PERIOD – OPTIONAL TASK – TIER 4 HELP DESK SUPPORT | $ | ||||
| TOTAL BASE PERIOD WITH OPTIONAL TASKS | $ |
Instructions to Offerors: This competition will be accomplished in two steps. In Step One the Offeror shall complete the MACM MVP Prototype Demo. Upon receipt of the Offeror proposal, specific instructions will be provided to each Offeror regarding the MACM MVP Prototype Demo.
- INTRODUCTION
The Offeror’s proposal shall be submitted electronically by the date and time indicated in the solicitation via email to Contract Specialist Jessica Adamitis, Jessica.Adamitis@va.gov, and Contracting Officer Juan Quinones, Juan.Quinones@va.gov, in the files set forth below. The Offeror’s proposal shall consist of five volumes. The Volumes are I -Technical, II – Past Performance, III – Price, IV – Veterans Involvement, and V - Solicitation, Offer and Award Documents. The use of hyperlinks or embedded attachments in proposals is prohibited. Offerors are encouraged to register for the Vendor Portal in order to ensure timely transmission of their proposals to the Government.
WARNING: Please do not wait until the last minute to submit your proposals! Late proposals will not be accepted for evaluation. To avoid submission of late proposals, we recommend the transmission of your proposal file 24 hours prior to the required proposal due date and time. Please be advised that timeliness is determined by the date and time an Offeror’s proposal is received by the Government not when an Offeror attempted transmission. Offerors are encouraged to review and ensure that sufficient bandwidth is available on their end of the transmission.
- PROPOSAL FILES. Offeror’s responses shall be submitted in accordance with the following instructions:
a. Format. The submission shall be clearly indexed and logically assembled. Each volume shall be clearly identified and shall begin at the top of a page. All pages of each volume shall be appropriately numbered and identified by the complete company name, date and solicitation number in the header and/or footer. Proposal page limitations are applicable to this procurement. The Table below indicates the maximum page count (when applicable) for each volume of the Offeror’s proposal.
All files will be submitted as either a Microsoft Word 2010 (.doc) [if allowing Word documents] Microsoft Excel (.XLS) file or an Acrobat (PDF) file or compatible as indicated in the table. Page size shall be no greater than 8 1/2" x 11" with printing on one side, only. The top, bottom, left and right margins shall be a minimum of one inch (1”) each. Font size shall be no smaller than 12-point. Arial or Times New Roman fonts are required. Characters shall be set at no less than normal spacing and 100% scale. Tables and illustrations may use a reduced font size not less than 8-point and may be landscape. Line spacing shall be set at no less than single space. Each paragraph shall be separated by at least one blank line. Page numbers, company logos, and headers and footers may be within the page margins ONLY, and are not bound by the 12-point font requirement. Footnotes to text shall not be used. All proprietary information shall be clearly and properly marked. If the Offeror submits annexes, documentation, attachments or the like, not specifically required by this solicitation, such will count against the Offeror’s page limitations unless otherwise indicated in the specific volume instructions below. Pages in violation of these instructions, either by exceeding the margin, font, printing, or spacing restrictions or by exceeding the total page limit for a particular volume, will not be evaluated. Pages not evaluated due to violation of the margin, font or spacing restrictions will not count against the page limitations. The page count will be determined by counting the pages in the order they come up in the print layout view. A Cover Page, Table of Contents and/or a glossary of abbreviations or acronyms will not be included in the page count of the Technical Volume. However, be advised that any and all information contained within any Table of Contents and/or glossary of abbreviations or acronyms submitted with an Offeror’s proposal will not be evaluated by the Government.
b. File Packaging. All of the proposal files may be compressed (zipped) into one file entitled proposal.zip using WinZip version 6.2 or later version or the proposal files may be submitted individually.
c. Content Requirements. All information shall be confined to the appropriate file. The Offeror shall confine submissions to essential matters, sufficient to define the proposal and provide an adequate basis for evaluation. Offerors are responsible for including sufficient details, in a concise manner, to permit a complete and accurate evaluation of each proposal. The titles and page limits requirements for each file are shown in the Table below:
Step One
| Volume Number | Factor | File Name | Page Limitations* |
|---|---|---|---|
| MACM MVP Prototype Demo | N/A | N/A | N/A |
Step Two
| Volume Number | Factor | File Name | Page Limitations* |
|---|---|---|---|
| Volume I | Technical Approach | Tech.pdf | 30 pages |
| -- | -- | Resumes.pdf | None |
| Volume II | Past Performance | PastPerf.pdf | None |
| Volume III | Price | Price.xls | None |
| Volume IV | Veterans Involvement | VetsI.pdf | None |
| Volume V | Solicitation, Offer & Award Documents, Certifications & Representations | OfrRep.pdf | None |
| -- | Small Business Subcontracting Plan (Large Businesses Only) | SBSP.pdf | None |
STEP ONE:
(i) MACM MVP Prototype Demo:
Upon receipt of the Offeror’s proposal, the Government will provide the same specific written instructions to each Offeror regarding the MACM MVP Prototype Demo. Each Offeror will be given five business days from the date the instructions are provided to prepare and deliver the MACM MVP Prototype Demo to the Government pursuant to the written instructions. Offerors shall deliver a MACM MVP Prototype demonstration that is accessible by Government staff for up to two weeks after the MACM MVP Prototype Demo for Government evaluation. Offerors will not be given an opportunity to correct or revise the MACM MVP Prototype Demo after its delivery to the Government. The Offeror shall prepare the MACM MVP Prototype Demo without any interaction with the Government. All costs associated with the MACM MVP Prototype Demo shall be borne by the Offeror and will not be reimbursed by the Government.
STEP TWO: (i) VOLUME I – TECHNICAL FACTOR. Offerors shall propose a detailed approach that addresses the following:
(1) The Offeror shall provide a detailed technical approach to the development and provisioning of the VA MACM Minimum Viable Product (MVP) and container orchestration platform IAW with PWS Task 5.2 and 5.2.1. The Offeror’s approach shall include its planned consumption/use of VAEC-AWS cloud service credits and its proposed model of application federation across the VA Network. The Offeror shall specifically detail the AWS cloud services and cloud service quantities required to develop the proposed VA MACM MVP.
(2) The Offeror shall provide a detailed technical approach to mobile environment support to include operations and maintenance of the VA MACM; security including obtaining and maintaining its Authority to Operate (ATO) and Software Security Plan (SSP) and security incident management; incident resolutions; disaster recovery including backup strategy and data retention; release and development management; and change control IAW PWS Tasks 5.3 and 5.4.
(3) The Offeror shall provide a detailed technical approach for the migration of all mobile applications located in IBM Terremark to the VA MACM as well as all associated databases and supporting services to the VA MACM IAW PWS Task 5.2 and 5.2.2.
(4) The Offeror shall provide a detailed approach for the management and optimization of resources including development, migration, maintaining, dashboarding, labeling, and financial accounting of used services. The approach shall include monitoring the health of the VA MACM systems, dashboard solutions, and its individual alerting solutions for management, technical subject matter experts and system developers/maintainers.
(5) The Offeror shall provide a clear description of how it will ensure deliverable metrics/SLAs are achieved; and quality assurance.
(6) The Offeror shall provide a clear description of the management methodology that will be used for executing the effort. The Offeror shall provide staffing approach to include proposed Key Personnel redacted resumes pursuant to PWS Section 5.1.4 and details of how the proposed effort will be assigned within the Offeror’s corporate entity and among the proposed subcontractors. The Offeror shall provide a summary of how it and its subcontractors’ corporate expertise will be utilized to meet the requirements of the PWS.
(7) The Offeror shall provide the estimated level of effort for its approach for each firm fixed price task and subtask to include labor categories and associated hours for the Prime and any proposed subcontractors for the base and option periods of performance. Ensure the hours are correctly totaled for each period (base and option periods) and a grand total is submitted for all periods combined. Details are required to be included in the Technical Volume and shall not be provided through a redacted price volume.
(ii) VOLUME II – PAST PERFORMANCE FACTOR. Offeror shall identify major subcontractors, if any, as performing more than 25 percent of the total proposed price and that will be providing any of the following tasks: VA MACM platform development and application migration; mobile environment support; release and development management; and help desk support as described in PWS. The information provided for the prime Offeror and each proposed major subcontractor must include the entire company name, company address, Commercial and Government Entity (CAGE) Code, and Data Universal Number System (DUNS) Number.
Offerors may identify contracts for review within PPIRS and/or submit contracts (including Federal, State, and local government and private) (prime contracts, task/delivery orders, and/or major subcontracts) in performance at any point during the three years immediately prior to the proposal submission date, which are relevant to the efforts required by this solicitation. Areas of relevance include contracts/orders over $5M for operations and maintenance of cloud platform(s), migration of applications and databases to a commercial cloud, development and/or operation of a container orchestration platform, and utilizing automation/Infrastructure as Code. The Offeror shall provide the contract number; description of the work; period of performance; and point of contact phone number and email address.
(iii) VOLUME III– PRICE FACTOR
The Offeror shall complete the Schedule of Supplies/Services found in Attachment 006 – VA MACM Excel Price Evaluation Spreadsheet. The Total Evaluated Price shall be based on the information provided in the Excel Price Evaluation Spreadsheet. Section B.4 Price schedule is for informational purposes only for the solicitation.
Price Rounding Issue - The Government requires Offerors to propose unit prices and total prices that are two decimal places and requires the unit prices and total prices to be displayed as two decimal places. Ensure that the two digit unit price multiplied by the item quantity equals the two digit total item price (there should be no rounding).
(iv) VOLUME IV – VETERANS INVOLVEMENT FACTOR.
(1) For SDVOSBS/VOSBs: In order to receive credit under this Factor, an Offeror shall submit a statement of compliance that it qualifies as a SDVOSB or VOSB in accordance with VAAR 852.215-70, Service-Disabled Veteran-Owned and Veteran-Owned Small Business Evaluation Factors (DEVIATION). Offerors are cautioned that they must be registered and verified in Vendor Information Pages (VIP) database (http://www.VetBiz.gov) and must meet federal small business size standards for the NAICS code assigned to this solicitation at time of both proposal submission and at time of award.
(2) For Non-SDVOSBs/VOSBs: To receive some consideration under this Factor, an Offeror must state in its proposal the names of SDVOSB(s) and/or VOSB(s) with whom it intends to subcontract, and provide a brief description and the approximate dollar values of the proposed subcontracts. Additionally, proposed SDVOSB/VOSB subcontractors must be registered and verified in VIP database (http://www.VetBiz.gov) in order to receive some consideration under the Veteran’s Involvement Factor and must meet federal small business size standards for the NAICS code assigned to this solicitation at time of proposal submission and time of award.
(v) VOLUME V - SOLICITATION, OFFER AND AWARD DOCUMENTS AND CERTIFICATIONS/REPRESENTATIONS.
Certifications and Representations - An authorized official of the firm shall sign the SF 1449 and all certifications requiring original signature. An Acrobat PDF file shall be created to capture the signatures for submission. This Volume shall contain the following:
a. Solicitation Section A – Standard Form (SF1449) and Acknowledgement of Amendments, if any.
b. Any proposed terms and conditions and/or assumptions upon which the proposal is predicated. However, all Offerors are hereby advised that any Offeror-imposed terms and conditions which deviate from the Government’s material terms and conditions established by the Solicitation may render the Offeror’s proposal Unacceptable, and thus ineligible for award.
c. Offerors shall provide list of all subcontractors including company name, CAGE code and DUNS number.
d. Large Business shall submit a Small Business Subcontracting Plan (SBSP) IAW FAR 52.219-9 and VAAR 852.219-9. The Offeror shall include in its SBSP the extent to which the Offeror meets or exceeds the Government’s Subcontracting goals for this procurement, which are as follows: Service-Disabled Veteran-Owned Small Business (SDVOSB): 5.0% of the total contract value; Veteran-Owned Small Business (VOSB): 7.0% of the total contract value; Small Disadvantaged Business (SDB): 5.0% of the total contract value; Women-Owned Small Business: 5.0% of the total contract value; Historically Underutilized Business Zone (HUB Zone) Small Business: 3.0% of the total contract value. Any inability to meet the Government’s subcontracting goal(s) or if the Offeror is not proposing to subcontract it shall include detailed rationale to support the determination. If the large business does not have an approved Master Plan or approved Commercial Plan, then an Individual Subcontracting Plan must be submitted that includes an assurance that small businesses will be given the maximum practicable opportunity to participate in contract performance. This plan shall be submitted separately from the Small Business Participation information required above, which applies to both Large and Small businesses. The Subcontracting Plan is not a requirement for evaluation in source selection, but rather, a requirement for award to a Large Business and the Plan, as negotiated, will be incorporated into any resultant contract.
e. Screen shots of the proposed prices out of the AWS estimating tool for the date the estimate was developed.
Offerors are hereby advised that any Offeror-imposed terms and conditions and/or assumptions which deviate from the Government’s material terms and conditions established by the Solicitation, may render the Offeror’s proposal Unacceptable, and thus ineligible for award.
Basis of Award Any award will be made based on the best overall (i.e., best value) proposal that is determined to be the most beneficial to the Government, with appropriate consideration given to the following five evaluation Factors: MACM MVP Prototype Demonstration, Technical, Past Performance, Price and Veterans Involvement. This competition will be accomplished in two steps. In Step One the Offeror shall complete the MACM MVP Prototype Demo. The MACM MVP Prototype Demo Factor only, shall be evaluated by the Government on a Pass or Fail basis. Accordingly, all proposals shall be assigned a final evaluation rating of either Pass or Fail under this Factor. Only those proposals determined to Pass under the MACM MVP Prototype Demo Factor shall be eligible to participate in Step Two and be eligible for award. The Government will not further evaluate proposal submissions from Offerors who are found ineligible in Step One. All proposals that Pass the MACM MVP Prototype Demonstration Factor shall be subject to a tradeoff evaluation between the remaining four Factors, in order to determine which proposals represent the best value to the Government. Those four remaining Factors shall be weighted as follows: The Technical Factor is significantly more important than the Past Performance Factor, which is significantly more important than the Price Factor, which is slightly more important than the Veterans Involvement Factor. To receive consideration for award, a rating of no less than "Acceptable" must be achieved for the Technical Factor. The non-Price Factors, when combined, are significantly more important than the Price Factor. Offerors are cautioned that the award may not necessarily be made to the lowest Price offered or the most highly rated Technical proposal. The Government intends to award a single contract as a result of the evaluations.
In Step Two, if the Contracting Officer determines that the number of proposals that would otherwise be in the competitive range exceeds the number at which an efficient competition can be conducted, the Contracting Officer may limit the number of proposals in the competitive range to the greatest number that will permit an efficient competition among the most highly rated proposals in accordance with FAR 52.215-1.
Factors to Be Evaluated
Step One:
- MACM MVP PROTOTYPE DEMO
Step Two:
- TECHNICAL
- PAST PERFORMANCE
- PRICE
- VETERANS INVOLVEMENT
Evaluation Approach: All proposals shall be subject to evaluation by a team of Government personnel. There will be no discussions as part of Step One, and the Government reserves the right to award without discussions based upon the initial evaluation of proposals in Step Two. The proposal will be evaluated strictly in accordance with its written and demonstrated content. Proposals which merely restate the requirement or state that the requirement will be met, without providing supporting rationale, are not sufficient. Offerors who fail to meet the minimum requirements of the solicitation will be rated Unacceptable and thus, ineligible for award.
STEP ONE:
MACM MVP Prototype Demo Approach:
Upon receipt of the Offeror proposal, specific instructions will be provided to each Offeror regarding the MACM MVP Prototype Demo. The instructions shall contain the ground rules of the demo along with the items that will be evaluated on a pass/fail basis. All Offerors will be given five business days from the date instructions are provided to prepare the MACM MVP Prototype Demo.
The MACM Demo is intended to demonstrate the Offeror’s ability to establish a MACM MVP prototype running a High Availability Kubernetes Cluster (K8S). The MACM MVP Prototype Demo will be evaluated on a Pass/Fail basis. When completed the Government shall be able to execute the code and standup the MACM MVP Prototype Demo, build, test, and deploy the application and service prototypes from a code repository. There will be no discussions as part of Step One.
STEP TWO:
- TECHNICAL EVALUATION APPROACH:
i. Understanding of the Problem - The proposal will be evaluated to determine the extent to which it demonstrates a clear understanding of all features involved in solving the problems and meeting and/or exceeding the requirements presented in the solicitation and the extent to which uncertainties are identified and resolutions proposed.
ii. Feasibility of Approach - The proposal will be evaluated to determine the extent to which the proposed approach is workable and the end results achievable. The proposal will be evaluated to determine the level of confidence provided the Government with respect to the Offeror's methods and approach in successfully meeting and/or exceeding the requirements in a timely manner.
- PAST PERFORMANCE EVALUATION APPROACH:
The Past Performance evaluation will assess the relative risks associated with an Offeror's likelihood of success in fulfilling the solicitation's requirements as indicated by that Offeror's record of past performance. In this context, “Offeror” refers to the proposed prime contractor and all proposed major subcontractor(s). Offeror shall identify major subcontractors, if any, as performing more than 25 percent of the total proposed price and that will be providing any of the following tasks: VA MACM platform development and application migration; mobile environment support; release and development management; and help desk support as described in PWS. In either case, the prime contractor and proposed major subcontractor(s) will be assessed individually and the results will then be assessed in their totality to derive the Offeror’s Past Performance rating.
The Government will conduct a performance risk assessment based on the quality of the Offeror's relevant and recent past performance, as well as that of its major subcontractors, as it relates to the probability of successful accomplishment of the required effort. The Government will review past performance data available in the Past Performance Information Retrieval System (PPIRS) for efforts in performance at any point during the three years immediately prior to the proposal submission date, which are relevant to the efforts required by this solicitation. Areas of relevance include contracts/orders over $5M for operations and maintenance of cloud platform(s), migration of applications and databases to a commercial cloud, development and/or operation of a container orchestration platform, and utilizing automation/Infrastructure as Code. The Government reserves the right to obtain past performance information from any available source and may contact customers other than those identified by the Offeror when evaluating past performance.
The Government will review aspects of Price/Cost, schedule and performance. Areas to be evaluated may include but are not limited to quality, schedule, cost control, management small business subcontracting and regulatory compliance.
In the case of an Offeror without a record of relevant past performance or for whom information on past performance is not available, the Offeror may not be evaluated favorably or unfavorably on Past Performance.
- PRICE/COST EVALUATION APPROACH.
This is a hybrid Firm-Fixed Price (FFP)/Time-and-Materials (T&M), Labor Hours contract. The total evaluated price will equal the sum of the total evaluated Firm Fixed Price (FFP), the total evaluated T&M price, the total evaluated Labor Hours price and the estimated VA MACM platform price.
Firm Fixed Price: The Government will evaluate offers by adding the total of all FFP line item prices, including all options, and optional tasks. The Total Evaluated FFP Price will be that sum.
Labor Hour: The total evaluated Labor Hour price will equal the sum of the total labor price for the entire task order period, including options, and optional tasks. The Government will verify the Offeror’s calculation of the total proposed price. The proposed fully loaded labor rates will be incorporated into any resultant contract, as binding, Not To Exceed (NTE) ceiling loaded labor rates. The proposed fully loaded labor rates will be used for the prime and all subcontractors/team members in performance of this contract. The estimated labor hours are for evaluation purposes only and do not obligate the Government to award such labor hours.
T&M: The total evaluated T&M price will equal the sum of the total labor price and the total of the Materials (including associated Material Handling Costs) for the entire task order period, including options. The proposed Material Fixed Handling rates will be incorporated into any resultant contract as binding NTE ceiling rates. The proposed fully loaded labor rates will be incorporated into any resultant contract, as binding, NTE ceiling loaded labor rates. The proposed fully loaded labor rates will be used for the prime and all subcontractors/team members in performance of this contract. The Government will verify the Offeror’s calculation of the total proposed price. The estimated labor hours are for evaluation purposes only and do not obligate the Government to award such labor hours
VA MACM Estimated Platform Price: For evaluation purposes only, Offerors shall price their proposed estimated VA MACM platform price based on commercial AWS pricing. Based on the Offeror’s technical approach, each Offeror shall provide an accounting of all cloud services, cloud service quantities and projected cost of cloud services based upon AWS Commercial pricing. Offerors shall use the AWS Commercial cost estimation tools found here: https://aws.amazon.com/aws-cost-management/. Offerors shall not discount the AWS Commercial pricing.
For Offerors from HUBZone business concerns that have not waived the evaluation preference, a price evaluation preference will be applied in accordance with FAR 52.219-4.
- VETERANS INVOLVEMENT EVALUATION APPROACH.
In accordance with Veterans Affairs Acquisition Regulation (VAAR) 852.215-70 (DEVIATION), Service-Disabled Veteran-Owned and Veteran-Owned Small Business (VOSB) Evaluation Factors, the Government will assign evaluation credit for an Offeror (prime contractor) which is a Service-Disabled Veteran-Owned Small Business (SDVOSB) or a VOSB. To receive credit, an offeror must be registered and verified in Vendor Information Pages (VIP) database at time of proposal submission and at time of award (https://www.vip.vetbiz.gov) and must meet federal small business size standards for the North American Industry Classification System (NAICS) code assigned to this solicitation.
Non-SDVOSB/VOSB Offerors proposing to use SDVOSBs or VOSBs as subcontractors will receive some consideration under this evaluation Factor. Offerors must state in their proposals the names of the SDVOSBs and VOSBs with whom they intend to subcontract and provide a brief description of the proposed subcontracts and the approximate dollar values of the proposed subcontracts. In addition, the proposed subcontractors must be registered and verified in the VetBiz.gov VIP database (https://www.vip.vetbiz.gov) and must meet federal small business size standards for the NAICS code assigned to this solicitation at time of both proposal submission and at time of award.