SpaceWire DB Analysis Tool - FSW Workshop

SpaceWire DB Analysis Tool Using SQL to investigate anomalies on Parker Solar Probe David Edell JHU/APL Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 1 Overview Parker Solar Probe uses SpaceWire as the primary data interconnect between components. Key SpaceWire Design Features: Parker Solar Probe SpaceWire DB Analysis Tool

High bandwidth interface Time synchronization Majority of traffic is RMAP initiated by primary SBC Analysis of traffic (when available) can provide valuable insights especially when telemetry is insufficient or disabled. Monday, February 2 4, 2020 2 Overview Links: - 2x redundant 8-port Routers - 3x Single Board Computers with direct SpW interconnects - 19 total SpaceWire links Packets

- Over 1000 packets per second on primary link - Multiple SpaceWire protocols - Many user-data formats If issues arise, SpaceWire analysis is often the only tool available - Parker Solar Probe kept recorders in I&T as long as possible to aide anomaly investigations - For some issues, analysis is simple (ie: traffic stopped) - Typically analysis requires sifting through a large volume of traffic to find the packet(s) of interest Analysis can be of events days, weeks, or even months in the past. Finding a specific data point, or pattern of data over time, is at best difficult to identify by hand. Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 3 SpaceWire Analysis Fault analysis & debugging often requires analysis of SpW traffic SpaceWire Data Recorders allow traffic to be monitored without interfering with the system - Star Dundee Link Analyzer

Monitors a single connection Provides a UI with header (including RMAP) decoding PSP has one in each testbed on our busiest link to aide developers Limited to recording a few seconds of data on high-traffic links when commanded. Documented C API Available - 4Links SpaceWire Recorder Monitors up to 8 connections Suitable for continuous recording o Currently configured to automatically start a new file every 6 minutes o Developers can retrieve archived log files for any time to match reported anomalies. Command line tools to parse binary files (can be slow to decode) o Periodically splitting files is necessary to ensure practical analysis Each manufacturer has custom data formats & tools to view the data Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 4 Recorders in action

4Links SpaceWire Recorders with continuous recording used for I&T, Stress Testing, and other complex scenarios. Data is automatically recorded 24x7 File timestamps can be correlated with ground system Star-Dundee tool used primarily for development. - Greater availability than 4Links Recorders Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 5 Raw Data Logs Star Dundee Link Analyzer: RMAP Headers can be decoded in the UI.

Data can be viewed at the packet or character level Counters are useful for high level debugging. Data can be exported as a CSV file. 4Links SpaceWire Recorder: These are the first few entries in an ASCII log file that can approach a GB in size. Parker Solar Probe SpaceWire DB Analysis Tool

Monday, February 2 4, 2020 6 A better way Repetitive analysis tasks are ideal for automation For complex analysis, the ability to query datasets is important - How many packets per second meeting select criteria? - Is packet type X always followed by Y? - Show only RMAP messages for a given LA when the data contents match some criterion. SQL Databases are the ideal tool to allow such queries to be quickly built against a large dataset. - For developer usage, any SQL administrative tool can be used to quickly query the database - For end-user usage, developing UIs around a SQL database is simple should the need arise. Basic Data Ingest: Use a script to parse existing ASCII log files

and insert the data into a database. - Better Method: Directly parse binary log files or access device API Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 7 Implementation 4Links - A Perl script was provided by 4Links to directly parse the binary log files and output to the shell - The script was modified to insert into the MySQL database instead. - Testbeds with 4Links Recorders in place configured to automatically record traffic at all times. Files can be correlated with ground system timestamps Data is archived for future analysis Star-Dundee - The C API was leveraged to directly capture data and insert it into a MySQL

database. - Application can be configured to, for example, capture exactly 1 second worth of data as measured by a set number of timecodes. - Quick and easy to use for developers. It takes a few seconds to complete parsing upon completion. Requires manual setup to record Not suitable for long duration analysis - For advanced triggering options, the official UI can be launched to configure the device. If not overwritten using the C API, those settings will persist. Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 8 Common Schema A simple database schema: - spw_capture - Contains a unique ID and timestamp for each log set, along with optional label and other identifying information - spw_events Log each event seen. disconnects and timecodes logged directly

Data is combined into a single packet and output on the first EOP or EEP. Common SpW Header fields pulled out for optimum indexing. - Additional tables provide mapping of links and logical addresses to user friendly names. - Table views are used to consolidate lookups and save common queries. This schema was built for developer convenience, not speed. - Table views created to extract RMAP and CCSDS headers, as well as applicationspecific data. - These views use non-indexed binary substring queries that are highly inefficient. - No effort was made to optimize queries by extracting common data (views) into discrete tables. Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 9 Usage Examples

Analysis of generic issues - Identify & import logs into database. On select testbeds, the 4Links recorders are continuously logging in the background. - Select COUNT(*) to estimate link activity - Filter on DIS events to identify timestamps of unexpected behavior - Filter the full log based on anomaly time to fine-tune analysis. Debug source of data anomaly - Identify & import logs into database - Construct one or more queries to extract a specific message type (by logical addresses and/or internal data flags) pull out specific data points into user-friendly fields. - Optionally filter results based on values of interest. For example, only show DISTINCT values - Export results to CSV file for offline analysis in Excel for non-SQL users Theoretically, a web UI could allow any user to do this directly. Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020

10 Future Enhancements User interface for analysis without requiring SQL expertise - Ideally, one of the many open source tools can be configured to serve this purpose. Automatic import of SpaceWire recorder logs - Continuous recording may not be practical due to processing time. - A UI could provide a simple framework to queue parsing of data for a given time period and testers can be instructed to do so when logging a new issue. Database schema optimizations Automated analysis tools or techniques to find potential issues - Can Big Data analytics or machine learning algorithms play a role in discovering subtle issues? Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020

11 Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 12 Parker Solar Probe SpaceWire DB Analysis Tool Monday, February 2 4, 2020 13

Recently Viewed Presentations

  • Si Se Puede! Achieving Separation of Duties with SQL ... - PASS

    Si Se Puede! Achieving Separation of Duties with SQL ... - PASS

    Module Signing. Jason has permission to call SP. SP run under Jason's context but with elevated privilege. SP protected against tampering. Jason (non privileged login)
  • Understanding by Design - Professor Arce's ESL - Home

    Understanding by Design - Professor Arce's ESL - Home

    UbD is a way of thinking purposefully about curricular planning and school reform. It offers a 3-stage design process, a set of helpful design tools, and design standards -- not a rigid program or prescriptive recipe. The primary goal of...
  • Le contexte de la recherche - HAL archive ouverte

    Le contexte de la recherche - HAL archive ouverte

    # Le cadre de référence du Sujet singulier, décrit notamment par Carl Rogers, est analogique du punctum de la perspective du décor, c'est la place du Prince au théâtre de la Renaissance : Le Prince occupe la seule place qui...
  • Sphere - Revelation And Creation

    Sphere - Revelation And Creation

    The last chirp of every sparrow…the last flight of every sparrow…the last heartbeat of every sparrow…the last breath of every sparrow touches God! Regarding Luke 12:26, J. W. McGarvey wrote, "If you can not add one little moment to your...
  • SUPERIOR VENA CAVA SYNDROME - Mans

    SUPERIOR VENA CAVA SYNDROME - Mans

    The superior vena cava is vulnerable to obstruction due to the following factors: its strategic location in the visceral compartment of the mediastinum, surrounded by rigid structures (such as the sternum, trachea, right mainstem bronchus, aorta and right pulmonary artery).
  • Theatre History - Loudoun County Public Schools

    Theatre History - Loudoun County Public Schools

    - a crane used to show characters in flight or suspended above the earth. Most often used for the appearances of gods. It became grossly overused in the last part of the 5th century, especially by Euripides. He often used...
  • www.ccclgbt.org

    www.ccclgbt.org

    LGBTQIA Safe Space Decals (Faculty) Professional Development Fund. Promo materials for QTPOC Training. Associated Student Government. Food for Queer Cupcake Party. Food for Coming Out Forum. Diversity Fund. LGBTQIA+ History Month, SPECTRUM Conference, Rainbow Graduation Stoles, Safe Space Lapel Pins,...
  • The next step in tv: Microsoft Hololens™

    The next step in tv: Microsoft Hololens™

    Picture tube TV - The TV contained vacuum tube containing one or more electron guns, and a fluorescent screen which was used to view images. Mechanical TV - The TV had two wheels spinning at high speeds but in sync....