Takeaway
Can’t read the entire technical blog? Here’s what you need to know: Sift gives hardware teams a consistent way to work with telemetry by normalizing and governing data in a single platform. The Single Source of Truth (SSOT) makes data easy to query and trust, while the Single Pane of Glass (SPOG) lets engineers explore, compare, and share without switching tools. Sift also integrates with Grafana, supports machine learning (ML) workflows, and provides client libraries in Python, Rust, and Go. These libraries are built on a gRPC API, with a REST API available for simpler use cases.
Why structured telemetry matters
This technical blog explores how Sift transforms fragmented hardware telemetry into a unified and trusted platform for analysis and decision making. In particular, this blog focuses on the following two core Sift capabilities that enable this transformation:
- a Single Source of Truth (SSOT), which ensures that all telemetry, regardless of source or format, is normalized, structured, governed, and queryable in the same place and in the same way, and
- a Single Pane of Glass (SPOG), which provides a unified interface to explore, validate, and share that data.
The blog begins by discussing Sift’s three-layer architecture: storage, governance, and application. It explains how these layers work together to enable both SSOT and SPOG, using a real-world example of a hardware team analyzing telemetry from autonomous rover tests across varied terrains. The blog ends with Sift’s integration features, including a Grafana plugin, client libraries in Python, Rust, and Go, a REST API, and Parquet export for machine learning (ML) workflows.
Sift’s three layer architecture
To deliver a SSOT and a SPOG, Sift’s platform is built on a three-layer architecture: storage, governance, and application. Each layer plays a distinct role in transforming fragmented telemetry into a unified, explainable, and actionable system.
Storage layer: building the SSOT
The storage layer forms the foundation of Sift’s SSOT, ensuring all telemetry is structured, governed, and queryable in the same place and in the same way. Sift supports ingestion from a wide range of telemetry formats, including ROS, MCAP, CSV, Parquet, HDF5, uLog, MQTT, PLCs, gRPC streams, JSON, images, video, and logs. These formats reflect the realities of hardware systems that produce data through sensors, embedded devices, control software, and field buses. Once ingested, telemetry is normalized into a consistent structure. This allows data from different formats and systems to be queried and analyzed using a common approach. It removes the need for format-specific processing and supports unified inspection and analysis.
All telemetry is normalized and stored in Parquet format in a centralized location for high-performance querying and analytics. This design supports real-time analytics, enables cross-signal comparisons, and allows both historical and live telemetry to be used reliably across workflows. This governed and queryable storage layer is what makes the SSOT operationally possible.
Governance layer: making the SSOT trusted
The governance layer gives structure and accountability to Sift’s SSOT, making analysis both possible and reliable. Core logic such as rules and calculations is version controlled, enabling teams to track changes over time, revisit previous configurations, and reproduce results as needed. Each update is recorded with context, providing a clear view of how logic evolves. Access control ensures that only authorized users or groups can view, edit, or create rules and calculations, allowing teams to collaborate effectively while protecting critical logic from accidental changes. This balance between flexibility and control keeps engineering teams aligned. Lineage tracking connects every result to its source and computation path, so whether a rule triggers an annotation or a calculation generates a derived value, the logic behind it remains transparent. By applying this structure across all telemetry, the governance layer ensures that analysis is grounded in trustworthy and traceable data.
Application layer: enabling the SPOG
The application layer powers Sift’s SPOG, the unified interface engineers use to interact with telemetry in their daily workflows. Through this interface, teams can visualize time-aligned telemetry including signals, logs, maps, images, and video. Engineers can define rules to detect issues using real-time checks and validate results with automatic or manual annotations. The interface also supports comparing runs at scale, allowing users to overlay and analyze hundreds of test sessions or mission logs side by side. When a view is shared, it retains the selected signals, filters, time ranges, and annotations, so that collaborators see the same data in the same context. This makes it easier to align across teams and maintain clarity during review or discussion.
This level of interactivity is possible because the data powering the SPOG is fully structured, governed, and unified through the SSOT. The application layer turns that foundation into an interactive environment where insights can be explored, validated, and shared efficiently.
Applying SSOT and SPOG in real-world workflows
To understand how Sift supports engineering teams in practice, consider a hardware team validating autonomous rovers after terrain testing. Their objectives include detecting anomalies, analyzing behaviors, comparing test results, and sharing findings across teams. Sift enables each of these steps while ensuring that all telemetry and context remain anchored in a structured and governed SSOT. The following capabilities illustrate how that process unfolds through Sift’s SPOG:
Automate data checks: Rules
Sift allows engineers to define Rules that automatically evaluate incoming telemetry for conditions such as voltage drops, thermal spikes, or timing irregularities. These Rules are evaluated in real time during ingestion. When the criteria are met, they generate Annotations tied to specific moments in the data. Because Rules live within the SSOT, their logic is consistent, version-controlled, and fully traceable across all test runs.
Capture engineering context: Annotations
Annotations allow teams to preserve both automated detections and manual observations. Engineers can add Annotations directly to timelines to document important moments, such as a rover slipping on a wet slope or failing to respond to a braking command. Annotations can also be created automatically when Rules are triggered. Since they are stored alongside governed telemetry within the SSOT, Annotations contribute to a shared, traceable record that supports review, collaboration, and decision-making.
Explore telemetry in one interface: Multi-view visualization
Sift unifies charts, logs, maps, and event timelines into a single time-aligned view. Engineers can correlate actuator commands, GPS location, terrain gradient, and sensor outputs within the same interface. Because all data is structured and governed through the SSOT, this exploration is accurate, consistent, and repeatable across sessions.
Collaborate without losing context: Shareable views
Engineers can share direct links to specific plot views that include selected Channels, time ranges, filters, and Annotations. These links preserve exactly what the sender sees, allowing teammates to open the same view and contribute without losing context. Because the underlying data comes from the SSOT, all collaborators are aligned around the same structured and governed telemetry.
Integrations
Sift integrates with existing engineering tools to support visualization, data export, and programmatic workflows.
- The Grafana plugin allows teams to query structured telemetry directly from Sift’s SSOT into external dashboards, providing an alternative to the Sift workspace while maintaining consistency and governance.
- For machine learning (ML) and data analysis use cases, Sift supports exporting telemetry as versioned Parquet files, which retain schema and structure. Because this data is already normalized and governed within Sift, exported files remain consistent, analysis ready, and traceable, extending the SSOT into downstream workflows.
- Developers can interact with the platform using official client libraries in Python, Rust, and Go, which are built on top of the gRPC API. A REST API is also available for simpler integrations. These interfaces provide access to the same normalized and structured telemetry available in the workspace, enabling ingestion, querying, and logic definition through programmatic means without compromising the guarantees of the SSOT.
Conclusion
Sift gives engineering teams a single foundation to work with hardware telemetry by delivering an SSOT and a SPOG. The SSOT ensures that telemetry from any format is structured, governed, and consistently accessible. The SPOG turns that foundation into a unified interface where engineers can explore, compare, annotate, and share data without switching tools or losing context.
FAQs
How does Sift create an SSOT from diverse telemetry formats?
Sift ingests telemetry from a wide range of formats, including ROS, MCAP, Parquet, MQTT, CSV, and others. All incoming data is normalized into a consistent internal structure and stored in a centralized location using a columnar format such as Parquet. This unified storage model ensures that telemetry can be queried and analyzed in a consistent and repeatable way, regardless of its original source or format.
What makes telemetry data in Sift trustworthy and reproducible across teams?
Sift applies version control, access control, and lineage tracking to signals, rules, annotations, and calculations. These governance features ensure that all data and derived insights are traceable to their origin, that changes are auditable, and that analysis can be consistently reproduced by any team member.
How does Sift support integration with external tools and workflows like Grafana and LLMs?
Sift integrates with Grafana through a dedicated plugin that allows teams to build dashboards from structured telemetry stored in the SSOT. It also supports exporting telemetry as Parquet files so teams can create versioned datasets ready for training or fine-tuning LLMs that can generate further insights.
Inside Sift’s architecture for structured telemetry
