---
title: Log Monitoring
slug: BMV0Q7EcRZBqi2I_6wUA0
description: Understand how to monitor errors in Ocient to maintain system health and prevent issues.
docTags: 
createdAt: Thu Jul 21 2022 17:05:13 GMT+0000 (Coordinated Universal Time)
updatedAt: Mon Mar 30 2026 18:52:43 GMT+0000 (Coordinated Universal Time)
---

:::hint{type="warning"}
Contact Ocient® Support for assistance before altering the system logging configuration in the `rolehostd.conf` file.&#x20;
:::

The Ocient Data Intelligence Platform™ logs events using these severities:

- `Error`
- `Warn`
- `Info`
- `Verbose`
- `Debug`
- `Extended Debug`

By default, the data warehouse creates the text log file `/var/opt/ocient/rolehostd.log` and logs events at the `Info` level and above.

Database logging supports extensive customization and integration with third-party tools. You can customize using the `rolehostd` config file. The database can return events in text, JSON, or Graylog® Extended Log Format (GELF) formats, or the database can return events in a file, to standard output (`stdout`), or to a UDP port. You can customize based on the log source.

## Log Formats

The default log format is text. These are some example log entries.

```Text
[2020-06-09T18:11:25.701594][ INFO][role001] roleHost: All roles started
[2020-06-09T18:11:25.701881][ INFO][role001] roleHost: Starting all endpoints
[2020-06-09T18:11:25.701925][ INFO][role001] roleHost: Starting endpoint 127.0.0.1:17900
[2020-06-09T18:11:25.704905][ INFO][role001] roleHost: Starting endpoint 127.0.0.1:17600
[2020-06-09T18:11:25.708437][ INFO][tcpU008] roleHost: All endpoints started
[2020-06-09T18:11:25.709236][ INFO][role001] host: ------------------------------
[2020-06-09T18:11:25.709286][ INFO][role001] host: Rolehostd successfully started
[2020-06-09T18:11:25.709305][ INFO][role001] host: ------------------------------
```

Each log line contains the time, severity, thread, and the source of the log (for example, `roleHost` or `host`).

JSON-formatted logs contain the same information but in a structured format.

```json
{"sys":"oc1.admin0", "timestamp":1591726285.701594684, "thread":"role001", "lvl":" INFO", "src":"roleHost", "msg":"All roles started"}
{"sys":"oc1.admin0", "timestamp":1591726285.701881144, "thread":"role001", "lvl":" INFO", "src":"roleHost", "msg":"Starting all endpoints"}
{"sys":"oc1.admin0", "timestamp":1591726285.701925519, "thread":"role001", "lvl":" INFO", "src":"roleHost", "msg":"Starting endpoint 127.0.0.1:17900"}
{"sys":"oc1.admin0", "timestamp":1591726285.704905192, "thread":"role001", "lvl":" INFO", "src":"roleHost", "msg":"Starting endpoint 127.0.0.1:17600"}
{"sys":"oc1.admin0", "timestamp":1591726285.708437413, "thread":"tcpU008", "lvl":" INFO", "src":"roleHost", "msg":"All endpoints started"}
{"sys":"oc1.admin0", "timestamp":1591726285.709236006, "thread":"role001", "lvl":" INFO", "src":"host", "msg":"------------------------------"}
{"sys":"oc1.admin0", "timestamp":1591726285.709286683, "thread":"role001", "lvl":" INFO", "src":"host", "msg":"Rolehostd successfully started"}
{"sys":"oc1.admin0", "timestamp":1591726285.709305246, "thread":"role001", "lvl":" INFO", "src":"host", "msg":"------------------------------"}
```

GELF-formatted log entries follow the format defined on the [GELF via UDP](https://go2docs.graylog.org/5-0/getting_in_log_data/gelf.html) page. This is an example GELF format.

```json
{"host":"oc1.admin0", "version": "1.1", "timestamp":1591726285.701594684, "_thread":"role001", "level":"5", "_source":"roleHost", "short_message":"All roles started"}
{"host":"oc1.admin0", "version": "1.1", "timestamp":1591726285.701881144, "_thread":"role001", "level":"5", "_source":"roleHost", "short_message":"Starting all endpoints"}
{"host":"oc1.admin0", "version": "1.1", "timestamp":1591726285.701925519, "_thread":"role001", "level":"5", "_source":"roleHost", "short_message":"Starting endpoint 127.0.0.1:17900"}
{"host":"oc1.admin0", "version": "1.1", "timestamp":1591726285.704905192, "_thread":"role001", "level":"5", "_source":"roleHost", "short_message":"Starting endpoint 127.0.0.1:17600"}
{"host":"oc1.admin0", "version": "1.1", "timestamp":1591726285.708437413, "_thread":"tcpU008", "level":"5", "_source":"roleHost", "short_message":"All endpoints started"}
{"host":"oc1.admin0", "version": "1.1", "timestamp":1591726285.709236006, "_thread":"role001", "level":"5", "_source":"host", "short_message":"------------------------------"}
{"host":"oc1.admin0", "version": "1.1", "timestamp":1591726285.709286683, "_thread":"role001", "level":"5", "_source":"host", "short_message":"Rolehostd successfully started"}
{"host":"oc1.admin0", "version": "1.1", "timestamp":1591726285.709305246, "_thread":"role001", "level":"5", "_source":"host", "short_message":"------------------------------"}
```

The database always returns message content in the `short_message` field of the GELF format.

This table maps Ocient severities to GELF system log (syslog) severities.

| **Ocient Severity**          | **Syslog Severity** | **Syslog Numeric Value** |
| ---------------------------- | ------------------- | ------------------------ |
| `Error`                      | `Error`             | 3                        |
| `Warn`                       | `Warning`           | 4                        |
| `Info`                       | `Notice`            | 5                        |
| `Verbose`                    | `Info`              | 6                        |
| `Debug` and `Extended Debug` | `Debug`             | 7                        |

## Log Appenders

A log appender is the destination for log entries. The database can send log entries to a file, a UDP port, or to `stdout`.

When logs are sent to the file appender, they grow continuously unless you use the `truncate` option in the configuration. You can use tools such as `logrotate` to manage log files.

This is a sample `logrotate` configuration.&#x20;

```Text
/var/opt/ocient/log/rolehostd.log {
   rotate 5
   daily
   delaycompress
   compress
   copytruncate
   missingok
   minsize 1M
   maxsize 100M
}
```

Test the UDP appender using the Linux® `netcat` command.

```linux
$ netcat -ul 9000
```

## Log Configuration

An Ocient System supports various logging settings to meet different needs.&#x20;

Set log configuration settings in `rolehostd.conf`, a YAML file normally found in the system `/var/opt/ocient/` directory. You can change configuration settings to suit your system logging needs in this file.&#x20;

For information on checking the status of nodes, see [Statistics Monitoring](<./Statistics Monitoring.md>).&#x20;

:::hint{type="info"}
If you are using an Ocient Simulator, logging behavior and directories might differ from these descriptions.&#x20;

For details on how logging works in the Ocient Simulator, see [Simulator Logging](<./././Ocient Simulator.md>).&#x20;
:::

`rolehostd`**&#x20;Example**

This is an abbreviated `rolehostd.conf` example in YAML format. Normally, this file contains other system configurations, but this tutorial focuses on the `loggingConfig` section, which defines logging behaviors and scopes.&#x20;

```yaml
loggingConfig:
    '':  # Default scope (all logs)
        appender:
        logFilePath: /path/to/rolehostd.log
    appenders:
    - appender:
        allowLevel: warn
        logFilePath: /path/to/error.log
    - appender:
        allowLevel: warn
        format: json
        logFilePath: /path/to/error.json
  
    dump:  # Dump-specific logs
        allowLevel: debug
        appender:
            logFilePath: /path/to/dump.log
  
    query:  # Query-specific logs
        allowLevel: info
        appender:
            format: json
            logFilePath: /path/to/query.json
  
    security:  # Security-specific logs
        allowLevel: info
        appender:
            format: json
            logFilePath: /path/to/security.json

    network-logs:  # Example network logging
        allowLevel: info
        appender:
            type: UDP
            server: graylog.internal
            port: 12201
            format: gelf
            compressed: true
```

### **Scope Definition**

The top level of the logging configuration is the scope, which determines the corresponding log sources for each configuration setup.&#x20;

**Default Scope**

An empty scope string (`''`) applies its configuration settings to all log entries the system returns.&#x20;

**Named Scopes**

Named scopes apply to log entries where the first component matches the name. Useful scopes include:

- `query` — Contains information on all queries and statements executed on the system, including users, time, and other details.
- `security` — Contains information on all connections made to the system, successful or unsuccessful.&#x20;
- `dump` — Contains information on system failures.&#x20;

### `allowLevel`

The `allowLevel` parameter specifies which severity levels of logs to capture.&#x20;

```yaml
allowLevel: value
```

Severity `value` definitions are:&#x20;

- `debug` — Most verbose (includes all logs). 
- `info` — Includes informational messages and greater severities. 
- `verbose` — More verbose than the `info` severity.  
- `warn` — Includes warnings and errors only. 
- `error` — Includes only error messages. 

### `appender`

Appenders define the format and directory where the database sends log files. You can configure either one or many appenders for each scope. &#x20;

**Single Appender**

For a single appender, use the `appender` key.&#x20;

```yaml
appender: 
    key: value
    [ ... ]
```

**Multiple Appenders**

For multiple output destinations, use the `appenders` key, followed by a list of `appender` keys.

```yaml
 appenders:
 - appender:
     key: value
     [ ... ] 
 - appender: 
     key: value
     [ ... ]
 [ -...]
```

### `format`

Defines the output format of the log files of an appender.&#x20;

```yaml
appender: 
    format: format_type
```

Supported `format_type` values are:&#x20;

- `text` — Human-readable text format (default). 
- `json` — Structured JSON format.
- `gelf` — GELF format for integration with Graylog.

### `logFilePath`

Defines the destination directory for log files of the appender. The `directory` value is a file path for the appender log files, for example: `/path/to/rolehostd.log`.

```yaml
appender: 
    logFilePath: directory
```

### `type`

Determines whether the system submits error logs through `stdout` or to a separate server using a TCP or UDP protocol.&#x20;

```yaml
appender: # standard output example
    type: stdout
     
appender: # UDP example
    type: UDP
    server: graylog-server.example.com 
    port: 12201 
    chunkSize: 1024  # Optional 
    compressed: true  # Optional  
          
appender: # TCP example
    type: TCP
    server: 192.158.1.38 
    port: 12201 
```

`server`

:::hint{type="info"}
Required for TCP or UDP types.
:::

The destination DNS name or IP address where the database sends log entries.

```yaml
appender: 
    server: 192.158.1.38 
```

`port`

:::hint{type="info"}
Required for TCP or UDP types.
:::

The port where the database sends log entries. The default value is `9000`.

```yaml
appender: 
    port: 12201  
```

`chunkSize`

:::hint{type="info"}
Optional for UDP type.
:::

The largest packet size in bytes that the system sends to the server. The database discards log entries larger than this value.&#x20;

```yaml
appender: 
    chunkSize: 1024   
```

`compressed`

:::hint{type="info"}
Optional for UDP type.
:::

If you set this option to `true`, the system compresses packets. For details about log compression, see the [GELF documentation](https://go2docs.graylog.org/current/getting_in_log_data/gelf.html). Otherwise, the packets are uncompressed.

```yaml
appender: 
    compressed: true    
```

### `truncate`

If you set this option to `true`, the system truncates the log file when the system starts. Otherwise, the system appends new logs to the file. The default value is `false`.&#x20;

```yaml
appender: 
    truncate: true
```

### `redactions`

You can redact data in logs at the SQL statement level using the optional `redactions` configuration. Enabling redaction creates another version of the logs with redacted SQL text. The `redactions` configuration value is `statement` for the redaction of part of the statement.

```yaml
appender: 
    redactions: 
    - statement # Optional
```

`rolehostd`**&#x20;Log Redaction Example**

This abbreviated `rolehostd.conf` file configuration enables log redaction at the statement level for both the `rolehostd.log` and `query.json` log files.

```yaml
loggingConfig:
    ? '': 
    appenders:
    - appender:
        logFilePath: /var/opt/ocient/logs/rolehostd.log
    - appender:
        logFilePath: /var/opt/ocient/logs/rolehostd_redacted.log
        redactions:
        - statement
    query:
        allowLevel: info
        appenders:
        - appender:
            format: json
            logFilePath: /var/opt/ocient/logs/query.json
        - appender:
            format: json
            logFilePath: /var/opt/ocient/logs/query_redacted.json
            redactions:
            - statement
```

## Related Links

[Install an Ocient System](<././Install an Ocient System.md>)

[Set Up System Monitoring with the TIG Stack and Kapacitor](<./Set Up System Monitoring with the TIG Stack and Kapacitor.md>)

***

**Linux®*&#x20;is the registered trademark of Linus Torvalds in the U.S. and other countries.*

