Free Courses

[Solution]

Photo credit: Miguel Cuenca @Pexels

The ARINC 661 avionics standard, developed by Aeronautical Radio, Inc., is pivotal in modern Cockpit Display Systems (CDS): it enhances the efficiency, usability, and safety of aircraft cockpit interfaces.

With the increased sophistication of avionics systems, ARINC 661 has become complex, featuring almost 120 widgets in Supplement 9. Each widget has unique behavior and display, making it time-consuming to test an ARINC 661 Server implementation.

In this blog series, we will explore a streamlined approach for testing the ARINC 661 Widgets using the SCADE Test Automation Framework (TAF) for widgets implemented with Ansys SCADE Solutions for ARINC 661 Compliant Systems.

In this article, we will first discuss the benefits and challenges of automatically testing ARINC 661 Widgets. We will then cover the features of TAF and how it addresses these challenges. Finally, we will guide you through the setup of the environment and share a concise example of testing a widget.

So, let’s get started on our journey.

Quick Introduction to ARINC 661 Widgets and their test automation

ARINC 661 is a standard for defining interactive avionics display systems. Its intent is to minimize the effort and cost of improving cockpits as technology evolves (new avionics systems, new features on existing systems, new hardware in the cockpit). Widely adopted across the aerospace industry, it promotes interoperability between avionic systems and simplifies vendor integration.

It is composed of UAs (for the logic), a CDS (server implementing the Widgets defined in the standard), and a safe runtime communication protocol.

If you are not familiar with this standard, we invite you to check our entry-level introduction to ARINC 661 in this previous blog post.

The initial testing activities for a hand-coded CDS project with 94 widgets are estimated to require around 300 person-months, excluding verification activities. With ARINC 661 Supplement 9 specifying 119 widgets and this number continually increasing, the workload for testing future CDS projects will only grow.

The need for automated testing in the development of ARINC 661 Widgets cannot be overstated, as it offers numerous benefits that significantly improve the efficiency and reliability of the testing process. By automating tests, development teams can achieve faster results, which is essential in today’s fast-paced world.

Automated testing also aligns seamlessly with CI/CD practices, empowering developers to quickly identify and fix issues early in the project while ensuring that changes do not disrupt the implementation. It also supports Continuous Improvement initiatives by enabling teams to consistently improve test cases and test procedures, maintaining high-quality software while keeping pace with rapid development cycles. Additionally, incorporating automated testing with TDD (Test Driven Development) leads to better design and maintainable implementations.

Automated testing eliminates human error, providing reliable results every time it is run, removing the need for tedious manual scenarios, and freeing up test engineers for other critical tasks in the development process. This boosts morale and energy levels. By enabling reproducibility, automated testing facilitates multi-target testing, ensuring that the CDS system functions consistently across various embedded hardware configurations.

These benefits of automatically testing ARINC 661 widgets make it a crucial component of an efficient development cycle. However, it encounters several challenges in its implementation:

• Setting pre-conditions: Before executing tests, the widgets under test must be set to a specific state, which might be difficult using only the ARINC 661 communication protocol.
• Simulating user interactions: Testing widgets requires manual user interactions through pointers, keyboards, touch, or gestures, making automated execution challenging.
• Checking events and notifications: The events or notifications emitted by widgets must be captured in a non-blocking manner.
• Performing graphical comparison: To verify the rendered display of the widgets, testers must have a method for display verification that can be automated (e.g., a screenshot mechanism followed by an image comparison).
• Describing test scenarios: Testers must have a language or method that allows them to clearly define test scenarios with support for the points listed above.
• Developing a test framework: Ultimately, the testing team must develop a Test Framework that provides a communication channel between the host and the target while effectively addressing all the described challenges.

Automating Widgets testing with SCADE Test Automation Framework

SCADE Solutions for ARINC 661 Compliant Systems provides a Test Automation Framework (TAF) that includes control proxy functionality in the ARINC 661 Server and a Python library. This library enables encoding/decoding of ARINC 661 messages and exposes commands to interact with the control proxy.

The usage of TAF can address the issues mentioned above:

• Setting pre-conditions: a set of commands can pilot the server, its services, and its interfaces to bring a given widget to a given state.
• Simulating user interactions: a set of commands can simulate pointers, keyboard, touch, and gestures; this even supports scenarios with multiple concurrent pointers, which are hard to test manually.
• Checking events and notifications: a command can execute the server step by step and wait for events for a given number of cycles.
• Performing graphical comparison: a command enables screenshot capture, and the image is sent back to the Python program for later comparison (for example using the image comparison tool provided in SCADE); additionally, it is possible to know the given state of the server (e.g. which widget has the focus) for simplifying testing strategies.
• Describing test scenarios: test scenarios can be written in Python using a series of function calls to TAF commands, enabling the reuse of common scenarios across multiple tests.
• Developing a test framework: the TAF allows running an automated Python test script commanding a live ARINC 661 server.

ARINC 661 Server and TAF

Now, let’s see how we can set it up and test an ARINC 661 Widget.

Setting Up the Test Automation Framework

After installing Ansys SCADE Solutions for ARINC 661 Compliant Systems, the basic setup of the Test Automation Framework requires setting up your Python environment (requires Python 3.6 or above).

First, add the following values to environment variable PYTHONPATH to import the Python modules of the Test Automation Framework and SCADE Python API:

• %SCADE_INSTALL%/SCADE A661/PythonLib
• %SCADE_INSTALL%/SCADE/APIs/Python/lib

%SCADE_INSTALL% should either be replaced inline (e.g. with a path like C:/Program Files/ANSYS Inc/v251/SCADE) or set as a separate environment variable (i.e. called SCADE_INSTALL).

TIP: Alternatively, you can add these values to sys.path in your Python code before doing the import of Test Automation Framework, like for example:

SCADE_INSTALL_DIR = "C:/Program Files/ANSYS Inc/v251/SCADE"

import sys
sys.path += [
    f"{SCADE_INSTALL_DIR}/SCADE A661/PythonLib",
    f"{SCADE_INSTALL_DIR}/SCADE/APIs/Python/lib",
]

We then need to install the python dependencies listed in %SCADE_INSTALL%/SCADE A661/PythonLib/taf/requirements.txt with either one of the following commands:

• pip install -r "%SCADE_INSTALL%/SCADE A661/PythonLib/taf/requirements.txt"
• pip install colorama pillow

You can refer to the Documentation delivered with the product for more details (also available online).

Basic content of Test Automation Framework

The Test Automation Framework is a package consisting of several Python modules, including:

• a661test.py: This module contains functionalities for tests, such as simulating user interactions and performing screenshots.
• a661libgen_rev9_v1.py: This module contains the A661 Widgets API, describing each widget with its parameters and events. (Note: the numbers in the file name could be different, based on the version of ARINC 661 used).

Other files provide core functionalities (e.g., the basic class of a Widget) and utility functions (e.g., configuring the logging mechanism, generating Definition Files, or checking events).

Verification of correct setup

Before using the Test Automation Framework, let’s make sure it is properly configured.

To do so, we will need a simple Definition File (DF) that we will be loaded into the ARINC 661 Server to check that no error has been reported.

For the creation of the DF, we will use UA Page Creator to implement a specification with a single ToggleButton widget with ID 1 at position (1000, 1000) and AlternateFlag=A661_TRUE to see more clearly if this is pressed or not. We then generate a binary file UA_1.bin.

Using this binary DF, we write few lines of Python that will look like the following:

# 1. import needed modules

# 2. initiate ARINC 661 Standard, Test Framework, and logging mechanism

try:
# 3. start the server, load the DF binary, and establish a connection between TAF and Server

# 4. check that no error has been raised by the Server

# 5. Test case usually goes here

except TAFCommandError as error: 
    # report error while communicating with the Server
except AssertionError as error:
    # report error during checks in the test
finally:
    # stop the Server

Note that we protect the main part of the test inside a try / except / finally block so that when an error or a failed check is detected, we stop instantly the test and the server.

For the imports, we need the logging mechanism, the selected ARINC 661 standard, the widget we test (i.e. a ToggleButton), a Test Framework, and the class for errors reported by TAF:

import logging

from taf.a661libgen_rev9_v1 import Standard_rev9_v1, ToggleButton
from taf.a661log import log_console
from taf.a661test import TAFCommandError, TestFramework

We then initiate the ARINC 661 Standard, the Test Framework, and the logging mechanism (to display errors and warnings as well as information message starting by “CHECK”):

Standard_rev9_v1()
te = TestFramework()

logger = logging.getLogger("taf")
log_console(level=logging.INFO, infofilter=["CHECK"])

We will then enter the try / except / finally block to first start the server, load the DF, and initialize the connection through the Runtime protocol.

    A661_SERVER_PATH = "C:/Program Files/ANSYS Inc/v251/SCADE/SCADE A661/bin/A661Server.exe"

    te.start_server(A661_SERVER_PATH, df_list=["UAPC_Model/DF/UA_1.bin"], bg_exec=True)
    te.init_connection()

Also, to map the notifications received by the server to the corresponding widgets, we need to register the content of the DF; to do so, we first use Python to create a DF corresponding to our model (a ToggleButton with ID set to 1) and we can then register this DF to the TAF:

    df = te.create_df([ToggleButton(1)])
    te.register_df(df)

Finally, we can check that no error has been raised by first moving one step forward (to let the server enter runtime mode), pulling the notifications, and checking that none of them is a “A661_NOTIFY_EXCEPTION”:

    te.step(1)
    notifs = te.pull_block(df=df)
    exceptions = [n for n in notifs if n["kind"] == "A661_NOTIFY_EXCEPTION"]
    assert exceptions == [], f"Exceptions raised: {exceptions}"

If at least one exception has been found, the program will exit with the list of exceptions printed in the console (and the server will be also stopped); otherwise, we can print a small message to say that everything is fine:

    logger.info("CHECK OK: ARINC 661 Test Automation Framework is correctly set up.")

The full code for checking that TAF is correctly setup and the Server can be correctly loaded and connected is the following:

"""Example of a check of configuration for A661 Test Automation Framework."""

# 1. -- IMPORT THE REQUIRED MODULES --
import logging

from taf.a661libgen_rev9_v1 import Standard_rev9_v1, ToggleButton
from taf.a661log import log_console
from taf.a661test import TAFCommandError, TestFramework

# 2. -- INITIALIZATION OF TEST AUTOMATION FRAMEWORK --
# * Set up which ARINC 661 Standard to use and create a Test Framework
# * Set up the logging for the test
Standard_rev9_v1()
te = TestFramework()

logger = logging.getLogger("taf")
log_console(level=logging.INFO, infofilter=["CHECK"])

try:
    # 3. -- START THE ARINC 661 SERVER (Initialization Phase) --
    # * start the server with the DF as binary file
    # * set the connection with server
    # * register the DF (to decode the events sent by the server)
    # We use the ARINC 661 Server path from the SCADE installation directory
    A661_SERVER_PATH = "C:/Program Files/ANSYS Inc/v251/SCADE/SCADE A661/bin/A661Server.exe"

    te.start_server(A661_SERVER_PATH, df_list=["UAPC_Model/DF/UA_1.bin"], bg_exec=True)
    te.init_connection()

    # We need to create a DF object that contains the widgets to be tested
    # that will be used to map the events sent by the server to Widgets types.
    # ... we could also use `create_df_from_sgfx` function from `taf.a661libutil
    df = te.create_df([ToggleButton(1)])
    te.register_df(df)

    # 4. -- CHECKING NO ERROR RAISED --
    te.step(1)
    notifs = te.pull_block(df=df)
    exceptions = [n for n in notifs if n["kind"] == "A661_NOTIFY_EXCEPTION"]
    assert exceptions == [], f"Exceptions raised: {exceptions}"

    # 5. -- TEST CASE USUALLY GOES HERE --
    logger.info("CHECK OK: ARINC 661 Test Automation Framework is correctly set up.")

except TAFCommandError as error:  # Error during communication with the server
    logger.error(
        f"Error {error.error} raised for command {error.command}: {error.description}"
    )
except AssertionError as error:  # Error when performing checks during the test
    logger.error(error, exc_info=True)

finally:
te.stop_server()

Example of a Test Case on an ARINC 661 Widget

Now that we know that the TAF and the Server have been correctly set up, let’s look at a small example of using the Test Automation Framework to verify that a button click is correctly handled by our ToggleButton (i.e. the server records an event).

We will also take screenshots to record the graphical rendering.

For our test, we will:

• Import Path from pathlib to specify the path of the screenshots
• Import taf.a661testcheck as checks to use utilities for checking notifications

import logging
from pathlib import Path

import taf.a661testcheck as checks
from taf.a661libgen_rev9_v1 import Standard_rev9_v1, ToggleButton
from taf.a661log import log_console

We will also modify part #5 (“TEST CASE USUALLY GOES HERE”) as follows:

• Capture an initial screenshot by using function te.get_screenshot with a specification of the width, height, and destination of the screenshot (we need to use a Path object)
• Click on the button and confirm that the A661 Server sends an event related to the button being pressed: to do so, we need to:
  • send a button pressed in a location above the button with te.push_pointer,
  • move the server one cycle forward with te.step,
  • release the button at the same location with te.push_pointer,
  • let the server run until we receive a notification for a maximum of 10 cycles with te.step,
  • check received notifications with te.pull_block,
  • finally, check that we have a A661_EVT_STATE_CHANGE notification with value A661_SELECTED for widget #1, using utility function checks.check_widget_event
• Capture another screenshot for comparison with te.get_screenshot

The implementation for this part is as follows:

    # 5.1 -- SCREENSHOT BEFORE CLICK --
    # * take a screenshot before the click
    te.get_screenshot(
        width=5000,
        height=3000,
        user_unit=True,
        resulting_file_path=Path("screenshot_before_click.png"),
    )

    # 5.2 -- CLICK ON THE BUTTON (Runtime Phase) --
    # * press the a click on the button
    # * release the click on the button
    # * wait for 10 steps or until the server send a notification
    # * check that the button has sent a notification
    te.push_pointer(1100, 1100, "LEFT", "PRESSED", user_unit=True)
    te.step(1)
    te.push_pointer(-1, -1, "LEFT", "RELEASED")  # -1,-1 to keep same pointer position
    te.step(10, True)  # wait for 10 steps or until the server send a notification
    evt = te.pull_block()
    check = checks.check_widget_event(
        notifications=evt,
        event="A661_EVT_STATE_CHANGE",
        widget_id=1,
        expected_values="A661_SELECTED",
    )
    assert check, "Test Failed: Button did not send A661_EVT_STATE_CHANGE"

    # 5.3 -- SCREENSHOT AFTER CLICK --
    te.get_screenshot(
        width=5000,
        height=3000,
        user_unit=True,
        resulting_file_path=Path("screenshot_after_click.png"),
    )

    # The test is successful
logger.info("CHECK OK: Test Case is PASSED")

The full code for our simple Test Case is the following:

"""Example of a test for a button widget in an ARINC 661 application."""

# 1. -- IMPORT THE REQUIRED MODULES --
import logging
from pathlib import Path

import taf.a661testcheck as checks
from taf.a661libgen_rev9_v1 import Standard_rev9_v1, ToggleButton
from taf.a661log import log_console
from taf.a661test import TAFCommandError, TestFramework

# 2. -- INITIALIZATION OF TEST AUTOMATION FRAMEWORK --
# * Set up which ARINC 661 Standard to use and create a Test Framework
# * Set up the logging for the test
Standard_rev9_v1()
te = TestFramework()

logger = logging.getLogger("taf")
log_console(level=logging.INFO, infofilter=["CHECK"])

try:
    # 3. -- START THE ARINC 661 SERVER (Initialization Phase) --
    # * start the server with the DF as binary file
    # * set the connection with server
    # * register the DF (to decode the events sent by the server)
    # We use the ARINC 661 Server path from the SCADE installation directory
    A661_SERVER_PATH = "C:/Program Files/ANSYS Inc/v251/SCADE/SCADE A661/bin/A661Server.exe"

    te.start_server(A661_SERVER_PATH, df_list=["UAPC_Model/DF/UA_1.bin"], bg_exec=True)
    te.init_connection()

    # We need to create a DF object that contains the widgets to be tested
    # that will be used to map the events sent by the server to Widgets types.
    # ... we could also use `create_df_from_sgfx` function from `taf.a661libutil
    df = te.create_df([ToggleButton(1)])
    te.register_df(df)

    # 4. -- CHECKING NO ERROR RAISED --
    te.step(1)
    notifs = te.pull_block(df=df)
    exceptions = [n for n in notifs if n["kind"] == "A661_NOTIFY_EXCEPTION"]
    assert exceptions == [], f"Exceptions raised: {exceptions}"

    # 5.1 -- SCREENSHOT BEFORE CLICK --
    # * take a screenshot before the click
    te.get_screenshot(
        width=5000,
        height=3000,
        user_unit=True,
        resulting_file_path=Path("screenshot_before_click.png"),
    )

    # 5.2 -- CLICK ON THE BUTTON (Runtime Phase) --
    # * press the a click on the button
    # * release the click on the button
    # * wait for 10 steps or until the server send a notification
    # * check that the button has sent a notification
    te.push_pointer(1100, 1100, "LEFT", "PRESSED", user_unit=True)
    te.step(1)
    te.push_pointer(-1, -1, "LEFT", "RELEASED")  # -1,-1 to keep same pointer position
    te.step(10, True)  # wait for 10 steps or until the server send a notification
    evt = te.pull_block()
    check = checks.check_widget_event(
        notifications=evt,
        event="A661_EVT_STATE_CHANGE",
        widget_id=1,
        expected_values="A661_SELECTED",
    )
    assert check, "Test Failed: Button did not send A661_EVT_STATE_CHANGE"

    # 5.3 -- SCREENSHOT AFTER CLICK --
    te.get_screenshot(
        width=5000,
        height=3000,
        user_unit=True,
        resulting_file_path=Path("screenshot_after_click.png"),
    )

    # The test is successful
    logger.info("CHECK OK: Test Case is PASSED")

except TAFCommandError as error:  # Error during communication with the server
    logger.error(
        f"Error {error.error} raised for command {error.command}: {error.description}"
    )
except AssertionError as error:  # Error when performing checks during the test
    logger.error(error, exc_info=True)

finally:
    te.stop_server()

Now let’s have a look at the two screenshots we made respectively before and after the click:

From this code and its execution, we can notice the key elements of the TAF:

• ARINC 661 Server Control: The ARINC 661 Server is exclusively controlled by TAF, disabling user interactivity and executing commands step-by-step.
• Command Execution: All server commands are executed by the TestFramework class.
• Screenshot Mechanism: The TAF provides a convenient way to perform screenshots of the server for automatically testing the graphical rendering of the CDS

Explore Further

This blog post about the ARINC 661 Test Automation Framework provides an overview of the problems it helps solve, how to configure it, and a brief demonstration of its usage. More blog posts will follow to expand on how to create Test Cases and perform specific checks.

Meanwhile, you can also check the “SCADE Test Automation Framework Guidelines” in the Ansys online documentation for SCADE Products.

If you’d like to learn more about Ansys SCADE Solutions for ARINC 661 Compliant Systems, we’d love to hear from you! Get in touch on our product page.

About the author

Jean-François Thuong (LinkedIn) is a Software Validation Manager at Ansys. He excels in software development, testing, and team management. His areas of expertise include the SCADE product, with a particular focus on Scade One. He is fluent in French, English and Chinese.