Written by Lorren Stahl, Technical Writer and Clara Cirks, Marketing Specialist
CWS Inform Project Background
In 2018, AF CyberWorx began a project to assess current readiness management systems for 621st Contingency Response Wing out of Joint Base McGuire-Dix-Lakehurst, New Jersey. AF CyberWorx interviewed Graduate Training Integration Management System (GTIMS) users to understand user needs and assess the ability of the GTIMS. Our User Experience (UX) Designers solicited feedback on user pain points, user journeys, and needs of an ideal system.
Key Factors Discovered in GTIMS User Interviews
AF CyberWorx team members discovered three key factors during this series of interviews:
Readiness is a critical factor for squadron commanders.
Commanders rely on the readiness information they receive to be current and accurate.
Current systems do not collate information, making readiness assessment difficult because information needs to be verified through more reliable sources.
Additionally, data sets pulled from GTIMS were disparate and lacked actionable intelligence. As the stakeholder’s goal was to enable commander/staff readiness orchestration and expand capabilities of the system, further development of solutions was integral. Small business CyberWinter Studios was introduced to AF CyberWorx and project stakeholders through a Small Business Innovation Research (SBIR) program to further develop solutions for the platform. The SBIR program enables small businesses to engage in federal research and development for technological potential.
Current Stages of Developed Solutions
Following the discovery phase, CyberWinter Studios designed and developed a minimum viable product (MVP) that consolidates relevant data into a single, intuitive format. This product and partnership are still active, and currently, CyberWinter Studios has a Phase II SBIR award. CyberWinter Studios’ current product, CWS Inform, has potential for enterprise automation, as the platform and code provide an interactive dashboard, automated workflows, and normalized back-end data. Code from this project has also been pushed to Github for common use by developers looking for a platform to understand and measure force readiness. Thank you to our partner with CyberWinter Studios, Mr. John Grigg, for continuing to push efforts to engage crucial technology with warfighters.
NPS & AU Grassroots Distributed Innovation Sprint: An Interview with TSgt Daniel Hulter
Written by SSgt Austin Wiggins
To accelerate change within the Air Force and Space Force, we must be able to equip potential innovators with education that promotes innovation at the lowest levels.Recently, CyberWorx had the fortune of hosting representatives of; Air University (AU), Naval Postgraduate School (NPS), Army 75th Innovation Command, Defense Innovation Unit, Army Space and Missile Defense Command, U.S. Navy, MIT Lincoln Laboratory, Tecolote Research, and Lokahi LLC. Despite being from disparate groups, they had the common goal of fostering grassroots and distributed innovation through education and connecting academic initiatives to address operational problems. The common understanding in the group; something had to change.
To uncover what made this session interesting and to highlight some hopes for the future of this project, I interviewed the facilitator of the event, CyberWorx member TSgt Daniel Hulter.
What were your expectations for the group going in?
I wasn’t sure what to expect from this group. It was clear they had a lot of topics they wanted to address regarding educational institutions’ role and needs as part of the defense innovation ecosystem. What wasn’t entirely clear was which of these topics were universally felt or what the priorities were. So, I knew that one of the first things we had to accomplish was getting on the same page about what needs were shared and which were more confined to a particular group.
One thing I did expect was that the conversation needed to be driven forward in order to move from observation and analysis to action. We generally have a high appetite for exploratory discussion, especially those in academic circles, and that can make the transition to concrete action challenging. Some participants advocated for less structure to the session so that loose discussion was allowed to happen freely for longer periods while they were together, something we attempted to balance against the desire for clear courses of action.
In what ways were those expectations positively subverted?
The group performed very well with the more structured portion of the workshop, which I won’t say subverted my expectations, but one of the things I was nervous about was whether they would appreciate being time-boxed and asked to perform very specific analytical tasks.
There were particular exercises that the group took to extremely well, for example their exploration of the defense innovation ecosystem through the lens of analogous systems. I remember feeling relieved that I wasn’t having to goad or micromanage them to do the exercises, which can be the case with certain groups.
What were your thoughts about the group’s objective to enable grassroots innovation through education?
I was slightly caught off guard by the inclusion of the “grass roots innovation” language in their problem statement. I realized that had I been more involved in the crafting of that problem statement, I might have been better prepared to speak to it. Instead I simply put it in front of participants and saw how they chose to interpret and include it in their exploration of the topics that came up.
I do think that the inclusion of that language speaks to the motivation of some of the organizers of this event to try and approach innovation education from a new angle, looking at distributed effects rather than targeted, exclusive, role-based impacts. But one thing we did run into there was that some of the individuals who helped craft that language did not join us for the session itself, which further limited the degree to which that problem statement guided participants in the direction they took.
I think there’s an opportunity in the motivation within this community to do a bit more to try and reimagine what innovation education could look like. It feels like what we accomplished in this day and a half was some broad scoping and a picture of some of the primary systemic mental models that are driving the current shape of the education system. One of the NPS hosts mentioned a paradigm shift around how we even think about innovation education and that’s really interesting.
But this wasn’t a workshop with a precise focus, it was much more exploratory. That question of how we might subvert or mutate or evolve the existing model to create something that is more aligned with the need for ground-level, grass-roots innovation is something that I think deserves significant time and attention and might be an interesting next step.
What are some outcomes you think that the group came to?
This group first achieved a few rough sketches of the current perceptual foundations of innovation education development and delivery for AU and NPS. Using that as an anchor, they thought about ailments within that existing system. They also arrived at a few measures that might surmount impediments to their primary value being delivered to more of the force. The proposed measures they came to at the end of day one can serve as launching points for experiments for these organizations to run, either together or separately.
Much of the time spent together in this session was also spent in discussions. I am confident that these interactions had a positive impact on the participants as they navigated the issues that came up during exercises on day 2, which was almost entirely unstructured. One of the common themes I heard come out was that this exact type of thing–coming together in one place and attempting to make sense of disparate and shared experiences across the defense education community–was something that ought to happen more frequently and perhaps with an even wider variety of participants. I think this may turn out to be one of the more significant outcomes, as it has the potential for continued delivery of insights, increased alignment, and sense-making, and increasing the likelihood of success for experiments that emerge from engagements like this.
What would you like to see from the organizations in this group in the future?
In the future, I would like to see these organizations do a few things:
Come together more frequently and invite more participants in, both with the type of forum that we created with this event and with more continuous connection mechanisms facilitated by platform selection/development, community building, and community management.
Spend more time thinking about and critiquing the perceptual foundations of their current organizational structures and strategies and put some dedicated time into seeing how those structures might be reimagined and redesigned.
Move forward with the experiments identified in the session as potentially high-value by giving them time and space in their own design efforts.
For those organizations who are seeking to empower grassroots innovation, do you have any recommendations?
My number one recommendation is to identify what is within the adjacent-possible for those you are expecting to innovate, for your particular context.
Every organization, unit, and team in the military has its own set of unique conditions and constraints that mean that the approach to enabling innovation and the form that that innovation takes has to be adapted to that level, a task which requires significant time and skill. Rather than seeking to identify standardized adaptations that might work for every circumstance, or even focusing on primary constraints of the larger system, I think it makes more sense to widely teach the very few underlying principles and practices that are universal. To try and affect a culture of acceptance, safety, and experimentation, and spend your remaining energy and resources on enabling that discovery and adaptation to happen in all the places it needs to happen.
If we are talking about true ground-level innovation, speaking as someone who has spent a majority of my career at the tactical level, I think that means enabling people to be sense-makers and practitioners within their own context. Those practitioners can be plugged into more strategic and operational-level scaling and implementation efforts, so creating systems that transition and scale outcomes should be a priority for organizations as well. In my opinion, empowerment of grass-roots innovation doesn’t start with the question of “are we getting things across the finish line” from a top-level perspective. It starts with creating the opportunity for individuals to contribute and have actual impact on their immediate environment.
Another recommendation that I have is to tell more stories of failure. One thing that came up during our workshop was the fact that shame is a powerful motivator. We are never so innovative as when we are ashamed because we allowed a terrible disaster to occur. The most incredible transformations are possible when we actually reflect on the current state of things and the ways in which we are failing (either by choice or when forced by some kind of incident).
A mistake I see a lot of organizations making right now is telling too few stories about failure. We regularly identify the need to normalize and embrace failure within our culture in order to spur innovation, but we still don’t talk about it enough, especially in the context of innovation (I think largely because we feel the need to say we’re succeeding in order to keep getting funding and support from our organizations and leaders). Lots of efforts at enabling innovation are failing, and we need to be open and honest about that in order to create space and energy for the next pivot.
For the record I am guilty of this exact thing. Storytelling and marketing are crucial components of success in all ventures, especially innovation. Building coalitions requires that we convince others of our likelihood of future success, and claiming present or past success is a potential pathway to that, but it suppresses stories of failure that might allow us to be ashamed enough of the status quo enough to drive the change we need.
Do you have any closing thoughts?
It was wonderful to see disparate players from across the innovation ecosystem come together and navigate these difficult topics together, in both structured and unstructured ways, and I hope to see this exact type of thing happen more often.
Thank you Daniel Hulter, for taking your insights into the session. With your participation, we are one step closer to delivering real qualitative change to grassroots and distributed innovators throughout the Air Force.
The chemical company, BASF, used to have an advertisement that said in part, “…we don’t make the cooler, we make it cooler. We don’t make the jeans, we make them bluer,” to highlight how their company improved the experience, performance, or durability of products for the people who use them. Much like the ubiquity of the products presented by the advertisement, information technology now underpins nearly every aspect of daily life both at work and at play. In a corollary, AF CyberWorx exercises their human-centered/user experience design, lean startup, and agile toolsets on the application of technology to improve experience, performance, and efficiency of mission execution, preparation, and support, while helping prepare the Air Force for new/emerging technologies and aiding industry in better understanding operational needs.
Our Cyber Risk Ecosystem, presented in previous newsletters and currently in integration/deployment with NORAD/NORTHCOM, provides cross-domain risk awareness to help commanders understand and mitigate cyber threats to missions in other domains. The recent low/no-code initiative for 16th AF demonstrated the efficacy of equipping unit-level Airmen with tools to automate and improve their own processes such as aircrew training and mobility readiness and awareness, network account request automation, contracting officer/small business interactive marketplace, personnel functions, and training instructor scheduling. Our team has helped the F-35 JPMO improve mission communications planning, and we see the potential for that work to expand across multiple platforms and across the Joint Services.
At AF CyberWorx, innovation isn’t just about technology, but a balance of meeting user needs within organization constraints with the right technology to make Airmen more effective at the full spectrum of activities needed to deliver Joint capabilities from mission support through mission preparation and execution. At AF CyberWorx, we don’t make the Commander’s decisions, we help make them better informed and more quickly. We don’t load the cargo, we make tracking it more efficient and accurate. We don’t fly the mission, we make the planning easier, faster, and more precise.
–Lt Col Helgeson, Deputy Director
CyberWorx 2021 Quarter 1 Project Portfolio
The Air Force Test Center wants to accelerate the data processing pipeline to move useful, actionable test range data to decision makers faster. CyberWorx is working with Hill AFB’s EDDGE software team to design and develop MVP screens for an application that streamlines access to a centralized data platform.
ai cyber wingman
The Cyber Wingman project is focused on enabling an AI-based mission commander digital assistant that can compile and process large amounts of data into actionable information, including similar past events and recommended courses of action. The CyberWorx team is working with MIT and Lincoln Labs, planning and coordinating research and interviews.
f-35 enhanced ui
CyberWorx is providing expertise to a joint work force of Navy, Air Force, and Marines with a deep dive into task analysis as they update their mission planning software that aligns to a next-generation framework with a modern user interface.
USSF Enlisted talent development
CyberWorx is working with USSF to identify how to best develop fully qualified enlisted Guardians that will be ready to meet future Space Force challenges.
leading user experience for everyone (Luxe)
The Chief Experience Officer of the Air Force is spearheading an effort to improve the form and function of existing enterprise-wide applications. The LUXE project underscores the importance of UX when developing products and services and highlights the need for wider adoption of UX practices across the Air Force. The CyberWorx team is providing design consultation and resources for education and training on human-centered design methodology as well as the redesign of some of the Air Force’s most egregiously designed applications.
AF Spark Tank finalist, Kinderspot, will offer families enrolled in Child Development Centers the ability to sublease their child’s spot to other eligible families while temporarily out of the area, Airbnb-style. The CyberWorx team worked with industry partner Oddball to optimize the user experience through the research and prototyping phase.
CyberWorx facilitated a design sprint 1-2 March to investigate how to develop a Consolidated Resourcing Sight Picture for the USAFA Financial Management office that would enable effective use of resources and provide better decision support to the Commander.
mission assurance with spectrum
We recently hosted nearly 150 government and industry reps to answer the question of how we might use the variety of Electro Magnetic Spectrum options to provide mission assurance of DoD operations. CyberWorx partnered with the Air Force Spectrum Management Office, the 350th Spectrum Warfare Group and NineTwelve, a public-private partnership in Indiana, to run a 3-day virtual event to connect emerging Spectrum technology with DoD stakeholders/users.
weather ai explainability
The Weather AI project will include explainable AI in future weather forecasting models with an emphasis on overseas locations with limited weather radar coverage to develop high-fidelity and explainable products. We are teamed with MIT and Lincoln Labs researchers on approach and questions for user testing in the effort to better inform the development of next-generation forecasting tools.
the other airmen
The Other Airmen initiative (pilot #1) successfully concluded on 11 March as six Airmen “Citizen Developer” teams pitched their solutions to 16th Air Force Commander, Lieutenant General Timothy Haugh at the Rocky Mountain Cyber Symposium. We are continuing work with the Air Force CIO office and 16th AF to move forward with finding ways to provide the capability to Airmen across the enterprise.
ussf cyber officer force development
We are assisting the Space Force in developing a more agile, digitally-focused career development path for their cyber officers.
21st century drill
Twenty-two participants from across government and industry attended a virtual design sprint from 26 January to 19 February to refine problem areas from information gathered through Guardsmen surveys about the collaboration and communication challenges they face getting ready for their drill weekends and annual training. A working group will continue work on the feasible solutions developed during the event to build the Air National Guard drill weekend of the future.
airmen leadership qualities
CyberWorx is collaborating with HAF/A1H as they work to make improvements to the Air Force’s officer and enlisted Evaluation Systems. Through facilitating AF-wide focus group sessions, the data gathered will inform decisions for evaluation system transformations in the future.
usafa superintendent’s honor system review
CyberWorx facilitated a design sprint for the Superintendent-directed Honor System review. Six teams of participants identified 31 ideas to improve the System and encourage cadets to embrace the Honor System as an ideal to aspire to, building leaders of character for the future.
winter is coming
The 319th Reconnaissance Wing and the University of North Dakota have teamed up to build a culture of continuous innovation in the Airmen of Grand Forks AFB, ND. We facilitated a virtual education and design event for them, focusing on improving the living conditions for Airmen living in base dorms by creating an environment that encourages engagement, curiosity, creativity, and inclusion.
norad-usnorthcom (n-nc) innovation and culture
Government and industry participants explored how to increase the digital literacy of N-NC and create a culture that embraces an innovative mindset.
The Other Airmen experiment aims to demonstrate citizen developers from the Air Force and Army can create useful applications using low code/no code capability. The teams are transforming their use cases into working applications to present to the 16th Air Force Commander Lt Gen Timothy Haugh in March 2021.
24 Feb – The process to pull and compare information for a unit member – military or civilian – is labor-intensive. It requires data to be cross-referenced from multiple sources including spreadsheets and specialized systems. Currently, personnel acting as Unit Deployment Managers, Unit Training Managers, Commander Support Staff, and other positions need to collect this information, compare it for accuracy and currency, and route it up to appropriate leadership.
Two Citizen Developers are collaborating on a unit-level application that imports and aggregates complex spreadsheets such as the Unit Manning Document, Unit Manpower Personnel Record, Alpha Roster, Gains and Loss rosters, and Civilian Roster pulled from personnel systems. They can also manually modify personnel records using “detail screens” which displays specific information on an individual.
The application suggests changes and allows users to manually verify where changes or inaccuracies exist. User access to system data is role-based, granted as their account is setup by an administrator. Role-based access maintains data confidentiality while still providing appropriate access for specialized personnel to gather information quickly for mission-essential tasks.
The application is currently functional with access to real data. The Citizen Developers are adding additional features from their “want to have” list. They are also gathering more information to provide a more robust example for the March presentation. They would like to import data directly from system databases, but current bureaucratic controls require manual spreadsheet imports. This application is a perfect example of individuals identifying a need for a tool that reduces time- and labor-intensive tasks to improve efficiency and accuracy. The sleek design also highlights that even amateur developers can develop relatively sophisticated applications that perform well and look professional in a relatively short span of time.
31 Dec – Today’s Spotlight focuses on a Market Research and Solicitation tool designed for contracting officers to efficiently perform market research on a company submitting for a Federal Commercial Solutions Opening. Companies could upload pitch decks including a descriptive video and whitepapers to support the submission. The tool could foster continued engagement with an organization after an event or project is done.
Currently, data in the MVP is entered manually. It includes links to the beta.sam.gov announcement and any social media platforms the company uses. The citizen developer is seeking APIs and permissions to automatically pull company information using the DUNS identification number, which is a unique nine-digit number required for any company to register with the Federal government for contracts or grants. Other potential sources for automatic information include usaspending.gov for contract identification, the Air Force Installation Contracting Center business intelligence unit for Federal Supply Codes and Product Service Codes, and ready-made prospective sheets for the company. The end goal is for the tool to minimize manual entry and provide users current information.
The citizen developer is an Air Force contracting officer currently participating as an Education and Industry fellow as her full-time position. She built the MVP on her own time on weekends after completing her training. The short time-frame from idea to MVP demonstrates how quickly a citizen developer can develop a solution that improves their productivity.
10 Dec – Of the citizen developers volunteering their time and efforts, one has accelerated the development timeline and delivered a prototype. War Skills and Military Studies instructors need a more efficient and reliable way to schedule a complex class coverage. The current process makes a team of instructors unavailable for teaching while they manually build the schedule. That draft schedule is visually compared to each instructor’s leave schedule and the class schedule. If a mistake or unforeseen change occurs, the team has to rush to make updates and disseminate the new schedule to the instructors as quickly as possible.
The use case involves a scheduling application that allows a single user to easily add classes, instructors, and locations as well as de-conflict instructors with scheduled leave. With the push of a button, the system will process the information and assign instructors to classes. Instructors won’t be scheduled if they’re on leave, nor scheduled to teach two classes at the same time. The system accounts for travel time between class locations. The application quickly produces an equitable schedule and allows the Scheduling Office the ability to easily and quickly make changes and disseminate an updated schedule within minutes instead of days.
That is the power of Low Code/No Code that The Other Airmen experiment is assessing for wider adoption. Citizen Developers know their pain points. Given the tools to develop their own solutions, DoD personnel can quickly improve unit efficiency.
11 Nov – Personnel responsible for member training requirements need a better way to track training data to reduce overhead and create a comprehensive picture of a unit’s ability to support the mission. Currently, commander support staff, unit training managers, and unit deployment managers have to sift through information from an array of sources including ADLS, TBA, IMDS, the Army’s DTMS, and various spreadsheets to piece together an accurate report. Users deserve a more efficient way to track training than cross-referencing 10 different systems to extract required data and manually input the information into a spreadsheet.
Several citizen developers are working on training tracker prototypes to streamline the current tedious process. An automated process will reduce the amount of time spent manually entering data, improve accuracy, and allow Airmen to focus their time and effort on other unit priorities.
Citizen Developers participating in The Other Airmen low/no code experiment made excellent progress towards functioning minimum viable products (MVP) in December. While the second in-progress review (IPR) in December had three demos, the third IPR doubled that with six demos.
The big rocks for the developers in December were clearing up lingering AFNet connection issues, expanding what capabilities they had available to them, and discovering APIs for data connectivity between their new applications and existing databases. The most successful capability demonstrated during the 12 January IPR was the seamless integration of DocuSign into an application for document package routing and signing. Other capabilities the developers are incorporating in their solutions include automatic account creation through CAC card registration and ranking data by confidence level. The breadth of use cases under development demonstrates the promise these platforms may hold for Airmen across the enterprise.
Several Citizen Developers are working on their solutions from home because of connection issues through AFNet. The ability to work on a solution from any location is important. We need to equip Airmen to work at their home station, while TDY, or even on deployment as long as they have an internet connection. We anticipate the barriers of going through AFNet will subside if low/no code is adopted as an enterprise approach for solution development.
As the teams pass the half-way mark of the experiment’s scheduled timespan, they transition from planning into development mode. The more mature solutions are now moving from “get it working” to “how can it be better” with more automation, a stronger user interface, or a broader range of database connectivity. The low/no code platforms and the Citizen Developers are exceeding expectations as they code future capabilities for their units.
OPTIMIS – In September 2020, we transitioned OPTIMIS to Kessel Run. CyberWorx further developed a USAFA cadet capstone concept from 2018, designing and coding a working prototype to upgrade the C-17 community’s standards and evaluation scoring system.
CWS Inform – Timely and accurate readiness information is a critical factor for commanders. In 2018, CyberWorx hosted a design sprint with subject matter experts, stakeholders, and users to examine the current GTIMS readiness management system including its ability to meet user needs and how personnel actually used the system. We partnered with a small business, CyberWinter Studios through a SBIR to create CWS Inform. Currently, a minimum viable product is in the user testing phase while CyberWorx continues to advocate for the deployment of this capability enterprise-wide.
COVID Response – When COVID-19 hit the Air Force Academy, we quickly responded. The entire staff moved into a telework environment and developed new procedures and tools to host events virtually to continue supporting our customers safely. The team stepped up to help our community, 3-D printing mask extenders for the medical and support staff at Ft Carson Evans Army Hospital. We continue to look for ways to improve our services in this new virtual world as commercial products rapidly upgrade to meet the demands of doing business virtually.
BizInt – We hosted a design sprint for an Air War College Blue Horizons team in January 2020 to examine the pre-deployment planning and contingency location needs of contracting personnel and logisticians. The solution addressed task completion for personnel in planning cell, SCO, and CCO positions. It also addressed two-way communication between all personnel involved in end-to-end logistics delivery. The basic design was transitioned to Headquarters AF, the office of the Secretary of the Air Force, with a SBIR phase 2 for further development.
Pi from the Sky – CyberWorx hosted a second Blue Horizons’ team in January for a design sprint to explore how to best use Raspberry Pi for a cost effective cyber tool in the battlefield for disruption, information gathering, and warfighter connectedness.
OpsAI – CyberWorx hosted a CyberNEXT event in January featuring Operational AI. Ten industry leaders in this discipline presented how their services were being used in the commercial sector. Afterwards, the CyberWorx team guided government users on how AI might help them with their use cases. The group was educated on a range of options for AI Proofs of Concept to tackle government needs.
ShOC n Awe – We hosted a collaborative discovery event at AFWERX-Las Vegas in March with twenty-five government and industry participants to determine near and mid-term organizational needs for the Shadow Operations Center (ShOC). The desired outcome; become the premier Joint All-Domain Command and Control Battle Lab for the future force. The goal is for the ShOC to become a development center for future distributed joint command and control without a permanent physical location. The ideas generated during the event were transitioned to Headquarters Air Force and AFWIC for further development.
ARCHER – CyberWorx partnered with Extreme Digital Development Group Enterprise (EDDGE) in April to take the “home-grown” ARCHER program to the next level with user-experience interviews, testing, and expert software engineering. ARCHER was a program designed to streamline some of the manually-intensive work done by trainers to build, edit, and process exercise scenarios for Intel Analysts. The designs and solution information were transitioned to EDDGE for further development.
6 Degrees of Kevin Beacon – The Air Force Rescue Coordination Center brought their manually-intensive coordination process to a CyberWorx design sprint in May to increase personnel effectiveness, reduce man-hours spent per incident, and reduce the decision-making time required to respond to incidents. Government and industry experts examined the requirements for a new cloud-based mission management tool for quicker, more accurate responses to civil search and rescue. We transitioned the information gathered and our models/recommendations for user experience designs for continued development by a company awarded a SBIR to develop the solution.
Airspace Defense Assessor – In May, CyberWorx hosted a virtual design sprint with personnel from NORAD-NORTHCOM and industry experts to improve information flow and the user interface for the AEISS platform update. The goal of the UI updates was to optimize the data flow and design a more intuitive UI for assessors to make critical, time-sensitive decisions. Information gathered during the sprint and designer suggestions were transitioned to the special program office as part of their ongoing development.
616 Convergence – Representatives from the 616th Operations Center and operational information warfare (IW) units met in June to improve situational awareness and understanding between mission partners to increase effectiveness. This challenge arose as the 24th and 25th Air Forces (AF) combined into the new 16 AF IW Numbered Air Force. CyberWorx transitioned the findings from the event with their observations to the 16 AF command structure for execution.
Early Warning Radar Sustainment– We hosted a virtual design sprint for Air Combat Command (ACC) stakeholders and industry experts in June to explore novel ways to sustain existing early warning radar capabilities for the next 20 years. Sustainment focused on maintainability, functionality, and form-fit factor. ACC used the information gathered during the event to compose an RFI (Request for Information) for industry commercial solutions.
USAFA Strong – Growing stressors during the COVID-19 pandemic prompted cadets and new second lieutenants from the Academy to approach CyberWorx for assistance in a USAFA-wide cadet project focused on fostering cadet connectedness. Government and industry advisors participated in a sprint to increase cadet engagement opportunities and discover innovative methods to increase mental health effectiveness at USAFA.
CSfC – In August, CyberWorx hosted a CyberNEXT event focused on Commercial Solutions for Classified (CSfC). Government and industry experts examined current capabilities and the unique challenges facing the DoD. By the end of the event, government representatives had formulated CSfC use case scenarios for experimentation.
ECO Talent Development – We hosted our first hybrid event in September with twelve in-person and four virtual attendees to capture the ideal attributes, training, education, positions, and experiences Expeditionary Communications and Cyber Officers should have at various stages of their career. The goal was to build a force development plan with the information gathered for the intentional development of future leaders.
The Other Airmen – During the development and transition of the OPTIMIS project, the CyberWorx team identified an opportunity for everyday Airmen to develop and implement solutions to their problems at the unit level. In September, a group of Citizen Developers from the Air Force and Army attended a discovery event to learn about the capabilities of commercial Low/No Code platforms. At the event, CyberWorx introduced Air Force initiatives already in place to develop and implement innovative ideas. Participants developed use case scenarios where Low/No Code would help them rapidly create and implement solutions to their challenges. From there, CyberWorx supported the Citizen Developers as they built their solutions into working prototypes. Presentation of the solutions to 16 AF/CC is planned for March 2021.
F-35 Link-16 and Multifunction Advanced Data Link (MADL) – Our team is working with SkiCAMP to improve the data link communication editor user interface. The goal is to eventually replace the outdated software tools to allow aircrew to accurately and intuitively plan data link communications across new and legacy platforms.
T-38 Kubernetes – We worked with Extreme Digital Development Group Enterprise and SkiCAMP on a Kubernetes-based software testing platform on a T-38 Talon. The experiment is a significant step towards real-time software upgrades during flight, enabling aircraft to land with better capabilities than when they took off.
KinderSpot – Air Force Child Development Centers (CDC) are an important benefit to military families. Our team is optimizing the user experience on an Airbnb-style application which would allow parents to use temporary childcare availability at base CDCs. A prototype is currently going through user testing for customer feedback.
Digital University – CyberWorx assisted the SAF/CN Chief Transformation and Chief Experience Officers in designing and building a roadmap for Digital University, destined to be a best-in-class online education application that offers high-value educational opportunities to Air Force and Space Force personnel for professional development. We fully transitioned the project to PEO BES/BESPIN for further development.
LUXE – CyberWorx doubled down on its vision to educate the Air Force on the importance of User Experience (UX) by working with the Chief Experience Officer of the Air Force. The Leaders User Experience for Everyone (LUXE) initiative is leading culture change across the Air Force by identifying and re-designing the user experience of three enterprise-wide applications while educating the developers on human centered design methodologies.
Weather AI Explainability – We are collaborating with the Air Force-MIT/Lincoln Labs AI Accelerator on a tool integrating explainable AI into weather prediction models in geographic locations without radar. CyberWorx and the AI Accelerator team are capturing user research to inform future design and development efforts. CyberWorx will deliver an MVP to the MIT/Lincoln Labs team the first quarter of CY21 and continue to iterate based on user feedback.
Cyber Wingman – CyberWorx is also collaborating with the Air Force-MIT AI Accelerator to refine the research and use cases for an AI tool that will compile and process information faster to accelerate accurate and effective decision making.
USAFA Honor Code Review – USAFA leadership is examining the honor code to determine if the policies need revision. CyberWorx is participating in this analysis, lending problem solving and human-centered expertise to the Academy-wide review.
USAFA Round Table – USAFA cadet groups are coming together to create a Round Table that provides a safe place for cadets of any nationality, race, gender, creed, religious affiliation, or group association to bring their concerns and suggestions. CyberWorx is performing an advisory role to the fledgling association as cadets build its structure and business practices to meet their objectives.
AF Gaming – AF Gaming gathered the most skilled Call of Duty gamers across the Air Force and Space Force to represent their branches in the Call of Duty Endowment Bowl. CyberWorx hosted AF Gaming leadership and the Space Force team with high-speed internet for the trans-Atlantic military eSports competition to promote connectedness among Airmen and Guardians across the force.
One of my Commanding Generals at the Joint Special Operations Command, GEN Scott Miller, instilled a culture in the command to “be whatever the nation needs us to be.” He recognized the need to proactively iterate because we face adaptive enemies, changes in leadership, accordion-like budgets, and other great unknowns. The Air Force needs CyberWorx to iterate for the exact same reasons. We do not want to fall into the Innovator’s Dilemma and rely solely on our past successes.
We were formed a few years ago by visionary USAFA alumni and Air Force senior leaders to focus on providing innovative solutions for cyber. Since then, the demand signal for our unique innovative methodologies to solve problems has increased drastically and broadened. Our portfolio has diversified from just cyber to C2, ISR, logistics, force development, missile warning, and even mental health to name a few.
We are going to continue to iterate so that we can be whatever the Air Force needs us to be (thank you GEN Miller). That includes a rebranding. The CyberWorx name has served us well, but stay tuned for a name change to reflect the awesomeness we provide to the Air and Space Force.
This past year saw tremendous growth at CyberWorx with new team members, an expanded customer base, successful transition of 19 projects, and over double the number of new projects spanning a wide range of topics from mental health to talent management and beyond.
Six new team member positions increase our ability to meet and exceed the growing needs of our customers and work load! For more details about members of our team, check out our team page.
Two more UX designers
Professor and subject matter expert in cyber engineering research and software development
A8P Program Analyst
FM Budget Analyst
A second Collaborative Project Order (CPO)
Word of mouth regarding our talent and expertise has gone viral. We’ve graduated from a few sporadic projects to repeat customers with enterprise-wide projects and COCOM priorities.
Air National Guard
US Space Force
System and joint project offices
Spark & project groups at individual bases
And many more!
The scope of our projects has grown well beyond cyber. We are a problem solving organization dedicated to shining a spotlight on the user in every initiative, no matter what the problem is. Some of the areas we’ve expanded into include:
Mental health problem solving with USAFA Cadets
UI designs for training and command decisions
Design sprints for talent management
Research into using explainable AI
The number of projects and requests has grown exponentially. In 2019, we averaged less than one event a month with a few non-event projects. We more than doubled that number for 2020. We successfully transitioned 19 projects this year from 2020 and earlier. The team worked tirelessly to identify partners and advocate the transition of solutions to make warfighters more effective and efficient. Check out the 2020 Project Review to go more in-depth on the good work we’ve done this year.
While 2020 has been a difficult year, CyberWorx continued to pivot and find better ways to help our customers. Our capabilities help organizations across the DoD when and how they most need it to address their most difficult problems. We will continue to provide high-quality support in 2021, confident that we will deliver value to the DoD and solve any problem thrown our way.
REBRANDING IN 2021
As our organization continues to evolve, positioning our branding to distinctly encompass what we do and who we serve is critical to our 2021 marketing strategy. Updated branding assets, including a new name and logo, will clearly represent our services and impact across the DoD enterprise. Strong, intentional branding elements will increase awareness of our multi-discipline services, reach additional warfighters and industry partners, and drive new collaborations and solutions.
No matter what we are named, our team will continue to focus on the user and the mission. Strategic branding is a vital component in setting ourselves apart as a top industry leader. As we transition into this new year, we are excited to reveal more details of this project!
QUARTER 4 PROJECTS RECAP
AF Gaming – We hosted AF Gaming and the US Space Force team for the CODE Bowl in our studio for a trans-Atlantic military eSports competition to promote connectedness amongst the force.
The Other Airmen – Citizen Developers volunteered for a low code/no code experiment to prove viability and capability for AF-wide application development solutions.
F-35 MADL and Link-16 – We are working with SkiCAMP to build a more user-friendly software tool to allow aircrew to accurately plan data link communications.
Kinderspot – We’re optimizing the user experience for parents AF-wide to use base CDCs with temporarily empty childcare slots, Airbnb-style.
USAFA Honor Code System Review – USAFA is reviewing the honor code to determine if any changes will help lower honor code violations.
LUXE – The Chief UX officer for the AF is improving the user interface of several AF-wide applications and increasing the appreciation and understanding of human centered design.
Weather AI Explainability – We are working with the MIT/Lincoln Labs AF AI Accelerator to build a tool that explains the confidence level for regional weather without radar coverage.
Cyber Wingman – Our work with the AI Accelerator has also provided the opportunity to build an AI-driven cyber intelligence capability.
21st Century Drill – The Air National Guard is designing the drill weekend of the future with more effective ways for Citizen Soldiers to communicate, collaborate, and increase readiness.
November was a busy month for our Citizen Developers participating in The Other Airmen experiment. It was primarily a month of preparation and learning. Citizen Developers and commercial partners formed relationships, ironed out technical issues, scoped their use cases, and developed work plans.
Database structure, security, and data access were big rocks for most of the developers. From commercial pitch decks for an interactive marketplace to storage of PDFs requiring signatures, developers solicited advice from AF experts to determine how best to store and disseminate sensitive data.
Meanwhile, a few teams played catch-up on platform training after working through access issues. Kudos to the military IT teams and the individual developers that worked through the firewall and permissions challenges quickly at this early stage! The teams can now focus on completing their concept maps including database and relational ties.
The real surprise came from a few teams that hit the ground running. They brought their commercial partner a well-developed concept map and plan, sprinted through training, and leapfrogged the timeline with a minimum viable product (MVP) already in hand.
December promises some exciting progress. The second in-process review (IPR) happened December 8th with three demos: two working prototypes and a solid wireframe. The other teams were poised at the first IPR to move into the building phase of their projects, and they have continued moving forward. The experiment is going extremely well! Stay tuned for future progress updates on this Low Code/No Code experiment as we evaluate this capability for use enterprise-wide!
The Situational Awareness Paradox And How Good UX Leads to Better Decisions with the Right Information (Part II)
By: Larry Marine and Dr. Dan Padgett
The Department of the Air Force is constantly looking to improve its situational awareness (SA). At CyberWorx, we’ve helped solve a wide range of problems. While each problem is unique, nearly all share a common theme of SA and data access. While data is a critical component to SA, simply displaying more information isn’t the answer. The data needs to be processed, normalized, and displayed in a way that improves human use.
In the last article, we explained the common problems we see in trying to “fix” legacy systems. (Read it here.) While these systems need to be improved, the opportunity exists to upgrade the entire system instead of building more technology into an outdated system.
Opportunities for Improvement
Just as a human-centered design perspective helps identify recurring issues, it also suggests four approaches for designing a system to correct them.
• Task-Oriented Design. We conduct user research for every project to understand what tasks and goals an Airman is responsible for. Focusing on tasks allows us to design solutions that provide functionality in the right way, at the right time.
Unlike feature-oriented design, where all features are available all of the time, task-oriented design doesn’t require knowing where each feature is located or when it’s appropriate to use it. Our goal is to design a system that will make someone just as successful on day one of their job as someone who’s been at it for several years.
• Task Management and Prioritization. Focusing on the task instead of the features allows the user to better manage and focus on each task, reducing the cognitive load and “switching cost” that comes with multitasking. Minimizing switching does not mean presenting all of the information at once. A well-designed system should assist users in prioritizing and managing their tasks so they switch tasks seamlessly and only when necessary.
• Mitigate Information Overload. Gathering, normalizing, correlating, and understanding data are all mentally demanding. Humans can only process so much data before they become cognitively overloaded. A system designed for users mitigates information overload by automating routine tasks – gathering, normalizing, and correlating as much information as possible – then presenting it to the user in an easy-to-understand format.
For instance, operators monitoring different sensors are primarily interested in trends and exceptions. Rather than overwhelming the operator with all of the available sensor data, the system can be designed to present only significant trends and exceptions. That doesn’t include omitting important information from the user. It’s organizing data into meaningful chunks related to that step of the task.
• Augmented Intelligence. The Department of the Air Force places a priority on keeping personnel in the decision-making loop. A fully automated system conflicts with that priority and isn’t feasible. Despite the advances in AI, machines still can’t mimic or replace human cognition. A better approach uses technology to support human thinking. The result is an intelligent application that uses computer strengths to complement human strengths.
Seizing opportunities for system-wide improvement will make personnel more efficient and decrease the workload on individuals while increasing their overall mission effectiveness. As the need for situational awareness grows and the amount of available data increases, delivering actionable information is more crucial than ever. We must define system requirements that put human needs as the focal point of those requirements. Hence the name Human-Centered Design.
The Situational Awareness Paradox And How Good UX Leads to Better Decisions with the Right Information
By: Larry Marine and Dr. Dan Padgett
The Department of the Air Force has a situational awareness (SA) problem. And no, the answer is not “we need more data.” Data, while a critical component to SA, is only a part of the problem. At CyberWorx, we often encounter solutions aimed only at acquiring and displaying more data, and we’re asked to design a visualization. The data is never the real problem.
From a human-centered design perspective, the systems we see are relics of the 1990s—and we’re not saying that because they lack the style of the latest visual design trends. Rather, the systems we see are technology-first approaches that saturate human cognitive capabilities. Adding more technology or data to the user’s task hinders his ability to complete the mission.
For example, a typical legacy system relies on the user to gather data through email, phone, and other disconnected sources. As the user gathers and processes this information, they log it–typically in an Excel spreadsheet custom built for the organization by a wizard who is no longer there to fix or update it. Once they’ve logged that information, the user can finally turn their attention to the task itself.
Often, something more urgent pops up before they can do that task. The user gets distracted and hopes they haven’t lost their train of thought when they return to the original task. Every time the user has to change tasks, technologies, or software, they incur a “switching cost.”  These costs add up, resulting in cognitive overload with potentially disastrous consequences.
Problems with Current Systems
Our discovery research on many projects reveals five recurring problems with legacy designs and proposed solutions:
• Pervasive Reliance on Humans. The systems we see rely on the user’s skill and knowledge (including how to use the system), both of which are highly variable. As a result, the success of the mission varies and cannot exceed the skills of individual users.
• Discrete, Non-Integrated Technology. Current systems are often a patchwork of technologies that are not integrated. This work of gathering, assimilating, and responding to information falls entirely on the user. While this “hands-on” approach to information sounds like it would increase SA, it actually puts the user at a perpetual risk for cognitive overload which can impact mission success.
• Processes Dictated by Technology. Many AF processes were developed as a response to the limitations of previous system capabilities. For instance, a conference call line might have been established due to communication difficulties, then process and protocol behaviors were established to support the conference call solution. These processes become sacrosanct, protecting the use of current technology—even when the technology is outdated, and it’s time to reevaluate and optimize (or remove) that process to promote new technology innovations. Keeping processes the same when updating technology doesn’t really change anything and limits the effectiveness of new technology. We refer to this phenomenon as automating current frustrations.
• “We Just Need More Training.” On its face, training sounds like a great way to overcome the above problems–more familiarity with the system should make it easier to use. However, we believe training should focus on the task, not using the system. If users need more training to use a system, it has poor design that takes attention from task completion.
• Automate, Automate, Automate. Automation can improve productivity by reducing the time to receive and view information. However, too much automation decreases efficiency. Without increasing overall comprehension, there’s only so much automation can do before the speed of information presentation overloads the user. (Have you ever tried to sip from a firehose?)
We can’t continue solving problems the same way we did in the 90s. We have the opportunity and ability to improve the entire system – not just add more technology – using human-centered design. The CyberWorx perspective with human-centered design not only calls attention to the recurring problems we see, it also suggests opportunities for improvement. We’ll discuss those opportunities in part II of this article. (Read it here)