Team Comments – Q3

Team Comments – Q3

What does an AF CyberWorx “win” look like? As we touched on in previous newsletters, we use our unique blend of user experience and modern problem solving/product development methods to tackle a wide range of operational and organizational challenges.  The solutions can range from a white-paper strategy/policy recommendation, re-engineered business processes, to technology (software/hardware) solutions or a combination of the three. 

We had three major wins in quarter three representing how we work to support Air Force solutions:

  •  Early Warning Radar Sustainment: As we supported Air Combat Command’s legacy radar sustainment effort, we not only assisted operators, maintainers, and sustainers in bringing together different viewpoints of the challenge, we combined a common future vision with acquisition strategies to engage industry to achieve the optimal solution for the Air Force. 
  • Six Degrees of Kevin Beacon: In our work with the Air Force Rescue Coordination Center, we helped them develop the concept for process flows and a software tool to streamline identification and response tracking to emergency beacons across the continental United States.  Additionally, we helped prepare them to submit for Small Business Innovation Research (SBIR) funds via AFWERX/AFRL, bringing together multiple partners in the AF innovation ecosystem and multiply AFRCC’s initial development investment. 
  • OPTIMIS: The originally airlift-focused flight evaluation app developed in a project originally taken up by Academy cadets took a different turn as we recognized complementary flight scheduling efforts under way by Kessel Run, another partner in the AF innovation ecosystem.  We transitioned our user-tested, minimum viable product into their portfolio to broaden its scope across all mission platforms, allow for full mission support integration, and provide for out-year sustainment. 

What all our solutions have in common is that through our end-user focused processes, we re-imagine how people, organizations, and technology interact to best accomplish and support the Air Force mission.  When we help with solutions, we don’t just stop with an idea or concept to solve a problem. We support with strategies and coordination to identify and engage partners for development and transition to sustainment. 

Successful outcomes from AF CyberWorx engagements are as diverse as the challenges they answer but always focus on the end-user to maximize mission impact and bring to bear our full range of academic, industry, and government partnerships to maximize transition potential. Resolve, Accelerate, Deliver! What would an AF CyberWorx-powered win look like for your organization?

Good News – OPTIMIS – Q3

Good News – OPTIMIS – Q3

As Lt Col Helgeson stated above, we’ve had multiple wins the last quarter. OPTIMIS is but one example and shows what we love to do: take a previously unaddressed end-user need, develop a prototype, and transition it to maturity and full sustainment.

In March 2018, the 21st Airlift Squadron (AS), Travis Air Force Base, California, reached out for help updating their home-grown pilot training and evaluation system, OPTIMIS. The unit built their database in MS Access years before and the developer had long since departed, leaving an unsupported system. Add to that their desire for more functionality and compatibility with the GTIMS system, and they were ready for some help.

Between November 2018 and November 2019, Academy cadets answered the call for help, using the real-world improvement project for a capstone course. As they worked on the improved app, they developed their skills in teamwork, user design fundamentals, and project management. By the final presentations, the cadets had created a mid-quality prototype that had already gone through the first phase of user testing at Travis AFB. Their time had come to graduate, however, so AF CyberWorx took the project and continued to code for transition.

AF CyberWorx carried OPTIMIS through more user testing, further refined the design, and found other programs following similar objectives. Kessel Run was already working on other mission planning and support applications, so we worked to transition a refined prototype to be included in their suite of matured applications. Recently, our attempts bore fruit as OPTIMIS transitioned for out-year sustainment. That’s our favorite part of a “win”: seeing the needs of end users being addressed as a solution moves from an idea through prototyping and testing to sustainment and implementation. Here’s to many more wins in the future.


At AF CyberWorx, we pride ourselves on our capability of Rapid Iterative Testing and Evaluation (RITE). RITE, by its very definition, iterates gathering requirements, designing prototypes, user testing those prototypes, learning what did and did not work from real users, and repeating throughout the design and development process. This increases the functionality and usability of the end product as well as improves the overall quality for the end user.

The RITE process has been used by our designers to great effect. With Optimis, instructor pilots track training details while still in the air with an intuitive mobile application. Commanders gain actionable situational awareness of their unit’s readiness and needs with the Automated Readiness Forecasting Tool. The Digital University maps goals and paths to success for cyber professionals, and the as-yet unnamed business intelligence tool gives contracting personnel a powerful application to easily capture and share important information across the operational area from planning to inspecting. RITE enables rapid improvement during the design and development of a project through user testing and immediate improvement.

What can RITE do for you?


AFCTM Email Newsletter AF CYberWorx


Last week, we had a slow start due to the snow storm on Tuesday; but, AF CyberWorx was able to recover and have a successful #AFCTM Sprint. Six teams split into groups of six and worked together to solve challenges related to Air Force’s Cyber Talent Management. 

We’ll have a press release and report prepared in a few weeks on the solutions that were proposed, but in the meantime, check out the photos taken during #AFCTM on our Flickr page. 

View Now!

Innovation Jan Email News 2019



From February 4-7, 2019 we’ll be at the Broadmoor in Colorado Springs for RMCS in booth 65. RMCS provides a national forum for industry and government to work together to help solve the challenges of cybersecurity, community cyber readiness, and national defense. Join us in the conversation to learn how you can work with us. 

Join Us ?

Networking Jan Newsletter AF CyberWorx



Connect with our government and industry personnel at Catalyst Campus for a networking happy hour event on an RMCS evening. Collider is an open house where guests can gain insight on AF CyberWorx projects of the past and future. We can’t wait to see you on Wednesday, February 6, 5-7 pm at Catalyst Campus Co-Lab Kitchen & Railyard.

RSVP Now ?

Upcoming Challenges AF CyberWorx Jan News


Join AF Cyberworx and the Air Force Research Laboratory in the Vice Chief’s Challenge. 

AF Challenge VCC

The Vice Chief’s Challenge is an open competition to solicit innovative ideas to tackle Air Force level problems. This year’s challenge will take on Multi-Domain Operations (MDO). Submissions must be entered into the Air Force IdeaScale before February 28, 2019

Join Us ?



“Our thoughts about an event can have a dramatic effect on how we go through the event itself.” –Martha Beck, life coach

Any time a group of people are about to embark on a new goal, there are expectations. As much as motivational quotes talk about having no expectations, they still exist in one form or another. A major part of the initial event planning is expectation management. Stakeholders have in mind what they think AF CyberWorx can do and will do. At the same time, the facilitation team has specific things they hope the stakeholders and participants will do. Sometimes the mystery continues on until the event itself. We hope this blog will work towards solving that mystery by setting realistic expectations.

When stakeholders request an event, they know the status quo is not working. They and the users they represent all have ideas of what might “fix” everything. Dr. Dan Padgett, a user experience designer on staff with AF CyberWorx, explains that the facilitation team does not “validate solutions they’ve already thought of.” Instead, they help with a process of thinking differently that gets the team to an answer. The key word being process. A problem solving event does not start with the solution. That being said, some participants and stakeholders come with the expectation that the AF CyberWorx team is going to either provide the answer to their problem or enable the idea they already have.

Vel Preston, Head of Innovation Design, explains it this way: “You scope a problem one way, and a lot of people come to the sprint who haven’t had the same background with new ideas. The group decides, ‘Well, we hadn’t considered that. Maybe we should scope this differently than we thought.’” The process AF CyberWorx guides event participants through leads experts and industry partners to refine the problem and find an impactful solution.

What the team thinks they’re going to fix at first may not be what needs to be focused on. Larry Marine, Lead UX Designer, explains why it’s necessary to lead participants away from their initial expectations as soon as possible: “Folks come in with a strong tendency towards the symptoms without fully understanding the problem.”

That being said, the facilitating team members are also not subject matter experts themselves. As Dr. Padgett says, “sometimes it takes participants a bit to realize that we’re there to facilitate the sprint, and that we’re not SMEs ourselves.” As designers, their specialty is facilitation and guiding teams to consider the user experience in their own designs. They are not experts in every field, nor are they experts on exactly what the stakeholders need to fix their problem. That’s why a group of experts from the field are brought together to solve the problem.

What AF CyberWorx cannot do for stakeholders is solve their problem. They will not immediately verify initial solutions before the process is followed, because they don’t know the current system well enough to say “yea” or “nay.” What we can do is assist a special focus team towards finding a desirable and feasible solution. We can provide facilities with a workspace, tools, and facilitators to make problem solving easier. We also provide networking with industry partners to broaden the knowledge and capabilities of the DoD talent pool. We facilitate improvement and growth, but the subject matter experts in the field are the real heroes that do the work.

Of course, expectations go both ways. Jayleen Guttromson-Johnson, Program Manager, lays out the AF CyberWorx expectations succinctly: “We expect full-time commitment while [participants are] in the design sprint. No email, phone calls, or disappearances. Also, be open to living with the uncomfortable. Our process isn’t like what most people are used to, so just trust the process.”

Stakeholders request AF CyberWorx facilitation and capabilities. We’re here for the team’s success. Let us help turn problems into solutions that go beyond simple expectations.



Written By: Author

What do hotels, food, security, and video recording have in common? They’re all part of the chain of logistics that happens behind the scenes before a successful event. The AF CyberWorx logistics guru is Cheyenne Ellis. She does more than just gather and send out a hotel list and coordinate access badges and video recording of the event outbrief. She also lines up transportation into the secure location where events are held, builds baskets with supplies for each breakout team, coordinates with our support team for food delivery, ensures buildings and elevators are accessible, and a myriad of other small details to let participants and the team focus their energy on the problem being worked during the event.

For her to be successful, though, she needs the cooperation of stakeholders and participants involved in the event. She says the most important thing she needs is for participants to fill out the requested information as quickly and accurately as possible. Otherwise, she may be unable to help, constrained by security or other deadlines. “We’re very strategic on what we ask for. We’re doing this because we need to.”

The place to give information is through the portal every participant is directed to at registration. The information provided allows Cheyenne to acquire security badges, lets her know if there are any food allergies (or preferences), as well as gives contact information in case there’s a last-minute change or weather issues to ensure participant safety and comfort. Without the proper information, we can’t get you on base, get you access badges, and through security to attend the event.

It’s critical that participants read the information they’re given before arriving on base for an event. It may involve what to bring (and not bring), base visitor hours, directions to the parking area, shuttle transportation, and how to navigate the maze of hallways to AF CyberWorx.

For stakeholders, the most important thing is to be involved and responsive, even if they are unable to attend the event. The best results happen when stakeholders give fully fleshed-out directions including their goals and a fully formed problem statement. Not only do these give the problem-solving team direction, they keep the stakeholder in the loop with what’s going on. The stakeholder is a critical part to shaping the way forward to a solution. Having involved stakeholders helps the event run smoother from the beginning all the way through approving implementing changes identified by the project teams.

AF CyberWorx does as much work behind the scenes as possible to make a design sprint run as smoothly as possible. Without cooperation from participants, however, that work is difficult. Be prepared with your end of the logistics and we will be that much more effective at designing a solution!


Photo: Shot from Office Space

AF CyberWorx is focusing on Human-Centered Design for the month of October. It’s the secret ingredient of all that we undertake and can be an extremely valuable addition to most processes. Please follow us on social media to gain more insight into the value of Human-Centered Design and enjoy this week’s blog post by our lead UX/UI designer, Mr. Larry Marine!


Anyone familiar with the cult classic Office Space will immediately recognize the TPS report. In the movie, the TPS report represented a comedic nod to the ubiquitous report generator common to pretty much every office software system. But while they are common, report generators are one of the most poorly designed tools in the UX domain.

The next evolution of the report generator has been the equally common dashboard. That multifaceted screen festooned with neon progress bars; red, yellow, and green speedometers; and blinking lights; all shiny objects that attract your attention, but don’t really do anything for you. Though both of these tools have become common solutions in software and web design, they fail to solve the users’ real problems.

Ask yourself this simple question: What will the user do with the report or the dashboard? Will they read it? Look for something? What kind of something?

Users tend to look for two things in a report or dashboard: trends and exceptions. Why? Because these are the two things that demand attention and action. But what action?

A key design tenet that I promote at the Air Force CyberWorx is every design should focus on leading to a contextually appropriate action. Avoid relying on the user to understand the problem or figure out what action to take. Lead them to the right action.

Since every trend or exception may require a unique action, a good UX design would provide the right action(s) appropriate for each item. For instance, rather than merely reporting that a sales rep is not meeting his numbers, a sales management system could incorporate a codified sales formula and compare the rep’s actions to that formula, noting any deviations and alerting the rep to perform any missing steps that are known to improve sales activities.

Example: it may be known that 80% of all sales come from a rep’s top 10 customers, but only if the rep meets with those customers at least 3 times per year. The system may know that the rep has only met with 4 of those customers and then suggest that he meet with the other customers soon.

The point is that dashboards and report generators themselves don’t incite action. The users are left to figure out what to do on their own. That adds a lot of cognitive burden onto the users, and there’s little reason to believe that every user will have the same response to every anomaly. The real design challenge is to incorporate the knowledge to define the problem for the user and then perform the correct action to address the specific trend or exception and to promote the right action for each trend or exception.

Think how differently your job would be if your dashboard synthesized the data, presented actionable insights, and then provided buttons to effect the correct actions.

This action-oriented design approach is a key focus at AF CyberWorx. Good UX design is more than just putting a pretty face on a dashboard, it’s about inciting the right actions to solve a specific problem.

*The postings on this blog reflect individual team member opinions and do not necessarily reflect official Air Force positions, strategies, or opinions.


AF CyberWorx is focusing on Human-Centered Design for the month of October. It’s the secret ingredient of all that we undertake and can be an extremely valuable addition to most processes. Please follow us on social media to gain more insight into the value of Human-Centered Design and enjoy this week’s blog post by our lead UX/UI designer, Mr. Larry Marine!


A recurring sprinkler control problem for large spaces (parks, ball fields, etc.) involves adjusting the sprinkler schedule to accommodate special events. A 3-day baseball tournament requires more than just shutting off the water for 3 days. Watering at night makes the field soft and susceptible to damage, so that’s not an option, either.

A more successful approach involves a complex schedule of watering more than usual for 2 days prior to the event, sprinkling only a few minutes each night of the tournament, then watering heavily again after the tournament. Existing solutions require users to manually change the watering schedule each day in the complex irrigation control UI, which almost always results in a user error.

A knowledge-based approach would be to provide different watering templates for various types of events that could be invoked by the users. For instance, the user might merely indicate that a 3-day tournament will occur on such-and-such dates and then the system would automatically adjust the watering cycles before, during, and after the tournament, returning to the standard settings after the event. Moreover, the templates could be based on best practices gleaned from other users with similar field composition and irrigation equipment.

By providing a design that asks the user to indicate the event and the dates, we can provide cues that focus on the knowledge we can expect users to actually have. So rather than a complex set of features, the user only needs to select the event and the dates, and the system does the rest of the work.

As technology becomes ever more complex and pervasive, we cannot expect users to understand how to use every device with complete expertise. They may be familiar with some devices, but not all. Therefore, we must design device interfaces so that users can use them without having expertise for that device.

A common approach to UI design is to provide a cornucopia of features and expecting users to know which features to use and when to use them. Unfortunately, this is a failed premise evidenced by the frequency of user errors. This is even more true when users are distracted by other factors such as state of mind and physical limitations. For instance, a person can’t be expected to attend to a complex device with patience and clarity when faced with a panic situation or if they are temporarily impaired.

Beyond simple UX Design, knowledge design builds knowledge into a design to help the average user bridge existing levels of skill and knowledge to be more successful. The key is to identify the knowledge users need in order to succeed and finding ways to embed that knowledge into a design.

Successful techniques for designing knowledge into a UI include designing best practice approaches into the product, creating task-oriented designs, providing templates and intelligent defaults. These techniques require that the designer have enough in-depth knowledge of the domain such that they can provide salient cues for the users.

CyberWorx uses techniques specifically focused on identifying knowledge design opportunities within a problem domain. We would be happy to discuss these techniques with your team. Just ask.

*The postings on this blog reflect individual team member opinions and do not necessarily reflect official Air Force positions, strategies, or opinions.


AF CyberWorx is focusing on Human-Centered Design for the month of October. It’s the secret ingredient of all that we undertake and can be an extremely valuable addition to most processes. Please follow us on social media to gain more insight into the value of Human-Centered Design and enjoy this week’s blog post by our lead UX/UI designer, Mr. Larry Marine!


Among the more popular forms of user research, usability testing is often conducted to gain insight into the users’ perceptions of a design. Like any other tool, usability testing yields the best results when performed correctly. An incorrectly performed usability test can lead you down the wrong path.

A common test design approach is to include specific directives on how to use a product, such as “print out a receipt.” While this seems innocuous enough, it actually biases the users’ behaviors and consequently the results. If users think they know the objective, they inherently try to please the tester. It’s in our nature. Users may even ‘game the system’ (focus more intently than normal on the stated outcome) to try and please us.

Rather than telling the users to do something, try using a question that can only be answered by performing a set of tasks. The advantages of using questions are plentiful:

  1. As humans, we love to solve problems or answer questions. Using a question in a test creates an intrinsic motivation that elicits a more realistic behavior. If users know what you are looking for, they are likely to alter their natural reactions and focus more attention on that specific action.
  2. A technique to hide your intentions about what you are testing for is to ask a question about an aspect of the task that occurs after the thing you are focusing on. For instance, instead of telling the user to print a receipt, ask them, “Can you tell how many pages are included in the printed receipt?” They will print a receipt as part of the question without realizing that is what you are focusing on.
  3. Questions reduce the tension a user might feel to perform a task. If they are given a directive, they believe it can be done. Thus, if they cannot complete the task, they feel it is their failure, not the design’s. If they cannot answer a question, they can legitimately say they don’t know the answer, which is less stressful and does not influence their behavior.

Another test design failure involves unintentionally biasing users by providing an incomplete prototype that lacks the screens users would run into when they veer off course. It’s not necessary to have a complete prototype, but at least have the screens a user would likely run into for a common set of errors for the tasks you are testing.

In the real world, users won’t immediately realize they have made a mistake and will continue to click around a few more screens. This observed behavior gives real insight into how the users finally determine that they have made an error and then how they try to get back to where they made the error.

A prototype that only has screens and interactions for the happy path unintentionally informs the users when they have taken a wrong turn, thus interrupting their natural behaviors. This limits your ability to identify what cues the users are relying on to determine if they are going down the right path and how they recognize when they haven’t. A key tenet of usability testing is that you learn more from watching users making and recovering from mistakes than you do from them NOT making mistakes.

Another common mistake is to rely on usability testing as the only user research method. Usability testing is an excellent method for capturing evaluative insights but is not that accurate for providing generative insights. Watching people use your design only informs you about the existing design, not about truly innovative design approaches or unmet needs. These require more generative methods such as interviews or observations.

Usability Testing can be a very useful tool, or it can mislead you if performed incorrectly. Biasing the users with a poorly conducted test yields inaccurate insights leading to design changes that solve the wrong problems. There is a major difference between what users say and do, and you should always lean towards performance rather than user preference. You must strive to design your test to collect accurate information, otherwise you are just wasting your time and your resources.

The team at AF CyberWorx can help you plan your next Usability Test (or any user research) to gain the maximum benefit.

*The postings on this blog reflect individual team member opinions and do not necessarily reflect official Air Force positions, strategies, or opinions.

Recent USAFA and AF CyberWorx Graduate Flourishes in New Position

AF CyberWorx Post Graduate Update Series

AF CyberWorx reached out to former cadets who worked with the innovation organization on campus before graduating. We caught up with two alumnae and found out what they are doing now, how their experience in AF CyberWorx influenced them, and what today’s cadets can do to better prepare for a successful future. Read our first alumna feature today.

2d Lt Emily Snyder, graduating class of 2018, and political science major, is currently a graduate student at the American University in Washington, D.C. She is studying international affairs with a focus on U.S. foreign policy and cyber security as it relates to national security. After finishing graduate school, she will become an Air Force cyber officer. While a student at USAFA, Snyder and her teammates worked together to develop a humanitarian application that would enable refugees to access information and form social networks while awaiting news of their asylum status in new countries with the goal of reducing refugee uncertainty.

Although the project was not completed before Emily graduated, she did learn a copious amount about the app development process. The AF CyberWorx project gave her the opportunity to think less in a traditional bureaucratic way, and more in a “Silicon Valley,” freeform mindset. The ability to think outside the box without penalty has immensely helped Emily with her current course load as a graduate student and as an intern at the Pentagon. While working on the project, her team collaborated with industry leaders at Pivotal Labs who elaborated on the creative process of application development. Those same connections she made at AF CyberWorx are still helping her today; Emily is working with Pivotal yet again during her internship and has been able to bridge the gap among her peers, thanks in part to the human-centered design she learned and applied in the project.

AF CyberWorx also opened Emily’s eyes to the importance of cyber security and how deep the field is. She hopes that in the future she can aid in outpacing policy with the implementation of new technology. She believes that giving the Air Force the ability to no longer need to play “catch-up” would create better mission succession rates.

Emily has a few words of wisdom for the third- and fourth-year cadets to keep in mind as they continue on their path of success within the Air Force Academy.

Be sure to focus on cyber. As the Air Force becomes more technically savvy, there will be an influx in the cyber field and positions that will need to be filled. Cyber will eventually become the new backbone of the Air Force, which is something you can gather by just seeing how much it has grown in the last 10 years. If you want to be successful within the Air Force’s cyber field, focus on what AF CyberWorx can teach you now while you’re in school, and you’ll be better prepared. On the same note of being focused on your future, be sure to pay closer attention to today’s current events…The problems you are seeing today will be the problems you need to solve in the future.

Do you have a story you’d like to share from your experience with AF CyberWorx? Tell us all about it, and you, too, could be featured on our blog. Email Jacki Stewart at

Want to stay updated on all things AF Cyberworx? Be sure to follow us on FacebookTwitterLinkedIn, and subscribe to our emails!