---
title: "Mobile Telemetry Processor"
url: https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor/
---

# Mobile Telemetry Processor

We're still figuring out the spec for the mobile telemetry processor. The telemetry buffer is already an approved concept, but not the rest of the processor.

## [Telemetry Buffer](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#telemetry-buffer)

🚧 This concept is approved but not yet implemented in any SDKs, and it's still being validated. If something feels unclear, too complex, or doesn't work as expected, please open an issue or PR and tag @philipphofmann for review. Feedback and improvements are welcome while we confirm the approach makes sense. 🚧

For the common specification for the telemetry buffer, refer to the [Telemetry Processor](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor.md) page. This page describes the mobile-specific implementation of the telemetry buffer. The most important difference is that the mobile telemetry buffer is designed to minimize data loss when sudden process terminations occur, such as crashes or watchdog terminations.

The common requirements specify [data forwarding scenarios](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor.md#data-forwarding-scenarios), which mobile SDKs MUST implement. Additionally, mobile SDKs **MUST** adhere to this extra data forwarding scenario:

* When the application moves to the background, the TelemetryBuffer **SHOULD** forward all data in memory to the transport and stop the timer. The transport **SHOULD** keep its existing behavior, which usually stores the data to disk as an envelope. It is not required to call transport `flush`.

Each SDK environment is unique. Therefore, SDKs have three options to choose from to minimize data loss. As their number increases, the options get more complex. The first option is the simplest, and the last option is the most complicated. SDKs **SHOULD** implement the least complex option that is suitable for their environment.

## [1. Flush All Data](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#1-flush-all-data)

When the SDK detects a sudden process termination, it **MUST** put all remaining items in the telemetry buffer into one envelope and flush it. If your SDK has an offline cache, it **MAY** flush the envelope to disk and skip sending it to Sentry, if it ensures to send the envelope the next time the SDK starts. The telemetry buffer **MUST** keep its existing logic described in the [Telemetry Processor Specification](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor.md#specification) page.

Suppose your SDK can't reliably detect sudden process terminations, or it can't reliably flush envelopes to Sentry or disk when a sudden process termination happens. In that case, it **SHOULD** implement the [FileStream Cache](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#2-filestream-cache) or the [DoubleRotatingBuffer](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#3-doublerotatingbuffer). It's acceptable to start with this option as a best effort interim solution before adding one of the more complex options.

## [2. FileStream Cache](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#2-filestream-cache)

SDKs for which blocking the main thread is a nogo, such as Android and Apple, SDKs **MUST NOT** implement this option. They **SHOULD** implement the [DoubleRotatingBuffer](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#3-doublerotatingbuffer).

With this option, the telemetry buffer stores the data on the calling thread directly to disk. The SDK **SHOULD** store the telemetry buffer files in a folder that is a sibling of the `envelopes` or `replay` folder, named `telemetry-buffer`. This folder is scoped per DSN, so SDKs ensure not mixing up data for different DSNs. In the `telemetry-buffer` folder, the SDK **MUST** store two types of cache files:

* **`cache`** - The file the processor is actively writing to
* **`flushing`** - The file being converted to an envelope and sent to Sentry

When the timeout expires or the cache file hits the size limit, the telemetry buffer renames the `cache` file to `flushing`, creates a new `cache` file for incoming data, converts the data in the `flushing` file to an envelope, sends it to Sentry, and then deletes the `flushing` file. When the SDK starts again, it **MUST** check if there are any cache files in the cache directory (both `cache` and `flushing`) and if so, it **MUST** load the data from the files and send it to Sentry.

## [3. DoubleRotatingBuffer](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#3-doublerotatingbuffer)

SDKs **SHOULD** only consider implementing this option when options [1](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#1-flush-all-data) or [2](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#2-filestream-cache) are insufficient to prevent data loss within their ecosystem. We recommend this option only if SDKs are unable to reliably detect sudden process terminations or consistently store envelopes to disk during such terminations.

The telemetry buffer uses two buffers to minimize data loss in the event of an abnormal process termination:

* **Crash-Safe List**: A list stored in a crash-safe space to prevent data loss during detectable abnormal process terminations.
* **Async IO Cache**: When a process terminates without the SDK being able to detect it, the crash-safe list loses all its elements. Therefore, the telemetry buffer uses a second buffer, the async IO cache, that stores elements to disk on a background thread to avoid blocking the calling thread, which ensures minimal data loss when such terminations occur.

As the telemetry buffer **MUST** prevent data loss during flushing, it uses a double-buffering solution. The crash-safe list has two lists `crash-safe-list-1` and `crash-safe-list-2`, and the async IO cache has two files `async-io-cache-1` and `async-io-cache-2`. When `crash-safe-list-1` is full, the telemetry buffer stores any new incoming items in `crash-safe-list-2` until it successfully stores items from `crash-safe-list-1` to disk as an envelope. Then it can delete items in `crash-safe-list-1`. The same applies to the async IO cache.

### [Telemetry Buffer Files](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#telemetry-buffer-files)

The SDK **SHOULD** store the telemetry buffer files in a folder that is a sibling of the `envelopes` or `replay` folder, named `telemetry-buffer`. This folder is scoped per DSN, so SDKs ensure not mixing up data for different DSNs. The `telemetry-buffer` folder **MAY** contain the following files:

* `async-io-cache-1` and `async-io-cache-2` - The async IO cache files.
* `detected-termination-x` - The file containing items from the crash-safe list from a previous detected abnormal termination.
* `envelope-x` - The envelope that the telemetry buffer is about to move to the envelopes cache folder, so the SDK can send it to Sentry, where `x` is the an increasing index of the file starting from 0.

### [Receiving Items](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#receiving-items)

The telemetry buffer has two lists `crash-safe-list-1` and `crash-safe-list-2` and two files `async-io-cache-1` and `async-io-cache-2`. When it receives items, it performs the following steps:

1. Put the item into the crash-safe `crash-safe-list-1` on the calling thread.
2. On a background thread, store the item in the `async-io-cache-1`.

### [Flushing](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#flushing)

When the `crash-safe-list1` exceeds the [above described](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#specification) 1MiB in size or the timeout exceeds, the telemetry buffer performs the following flushing steps:

1. Store new incoming items to the `crash-safe-list-2` and `async-io-cache-2`.
2. Put the items of `crash-safe-list-1` into an envelope named `envelope-x`.
3. Delete the items in `crash-safe-list-1` and `async-io-cache-1`.
4. Move the `envelope-x` to the envelopes cache folder, in which all the other envelopes are stored, so the SDK can send it to Sentry.

The telemetry buffer stores the `envelope-x` not directly in the envelope cache folder because, if an abnormal process termination occurs before deleting the items `crash-safe-list-1` and `async-io-cache-1`, the SDKs might send duplicate items.

### [Abnormal Process Termination](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#abnormal-process-termination)

When SDKs detect an abnormal process termination, they **MUST** write the items in both `crash-safe-list-1` and `crash-safe-list-2` to the `detected-termination-x` file where `x` is the an increasing index of the file starting from 0.

When the process terminates abnormally and the SDKs can't detect it, the SDKs lose items from the crash-safe lists, which we consider preferable to blocking the calling thread, which could be the main thread. However, the SDKs don't lose items from the async IO cache.

So the SDK **MAY** lose items due to undetectable abnormal process terminations that occur immediately after receiving an item, but it won't lose items due to detectable abnormal process terminations.

```bash
// SDKs keep this log
Sentry.logger.info("We might crash now.");

crash();
```

### [SDK Initialization](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#sdk-initialization)

Whenever the SDKs initialize, they **MUST** check if there is any data in the telemetry buffer folder that needs to be recovered. They **MUST** perform the following steps when initializing:

1. Load all items from `async-io-cache-1`, `async-io-cache-2` and `detected-termination-x` into memory if they exist. When the application terminates normally, these files don't exist.
2. Deduplicate the items based on the IDs of the items and store the deduplicated items in the `envelope-x` file.
3. Create new `async-io-cache-1` and `async-io-cache-2` files, and delete the `detected-termination-x` file.
4. Now the telemetry buffer can start receiving new items.
5. Move the `envelope-x` to the envelopes cache folder.

### [SDK Closes](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#sdk-closes)

Whenever the users closes the SDK or the application terminates normally, the telemetry buffer **MUST** perform the steps described in the [Flushing](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#flushing) section and the SDK **MUST** delete all items in the `async-io-cache-1` and `async-io-cache-2` files.

## [Miscellaneous](https://develop.sentry.dev/sdk/foundations/processing/telemetry-processor/mobile-telemetry-processor.md#miscellaneous)

The telemetry buffer maintains its logic of batching multiple logs and spans together into a single envelope to avoid multiple HTTP requests.

Hybrid SDKs pass every log and span down to the native SDKs, which will put every log and span in their telemetry buffer and its cache when logs and spans are ready for sending, meaning after they go through beforeLog, integrations, processors, etc.