README.md
author Mahlon E. Smith <mahlon@laika.com>
Wed, 14 Feb 2018 17:27:29 -0800
changeset 6 1f366fc61592
parent 5 a1276c3d39eb
child 10 252cdb26f76b
permissions -rw-r--r--
Each incoming connection requires its own client socket.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     1
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     2
Netdata-TSRelay
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     3
===============
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     4
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     5
What's this?
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     6
------------
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     7
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     8
This program is designed to accept JSON streams from
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
     9
[Netdata](http://netdata.io) clients, and write metrics to a PostgreSQL table -
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    10
specifically, [Timescale](http://timescale.com) backed tables (though
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    11
that's not technically a requirement.)
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    12
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    13
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    14
Installation
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    15
------------
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    16
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    17
You'll need a working [Nim](http://nim-lang.org) build environment to
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    18
create the binary.
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    19
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    20
Simply run `make release` to produce the binary.  Put it wherever you
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    21
please.
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    22
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    23
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    24
Configuration
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    25
-------------
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    26
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    27
There are a few assumptions that should be satisfied before running
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    28
this.
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    29
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    30
### Database setup
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    31
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    32
You'll need to create the destination table.
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    33
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    34
```sql
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    35
CREATE TABLE netdata (
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    36
	time timestamptz default now() not null,
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    37
	host text not null,
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    38
	metrics jsonb default '{}'::jsonb not null
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    39
);
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    40
```
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    41
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    42
Index it however you please based on how you intend to query the data,
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    43
including JSON functional indexing, etc.  See PostgreSQL documentation
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    44
for details.
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    45
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    46
Strongly encouraged:  Promote this table to a Timescale "hypertable".
3
f34855556f4d Add timescale creation example.
Mahlon E. Smith <mahlon@laika.com>
parents: 1
diff changeset
    47
See Timescale docs for that, but a quick example to partition
f34855556f4d Add timescale creation example.
Mahlon E. Smith <mahlon@laika.com>
parents: 1
diff changeset
    48
automatically at weekly boundaries would look something like:
f34855556f4d Add timescale creation example.
Mahlon E. Smith <mahlon@laika.com>
parents: 1
diff changeset
    49
f34855556f4d Add timescale creation example.
Mahlon E. Smith <mahlon@laika.com>
parents: 1
diff changeset
    50
```sql
f34855556f4d Add timescale creation example.
Mahlon E. Smith <mahlon@laika.com>
parents: 1
diff changeset
    51
SELECT create_hypertable( 'netdata', 'time', chunk_time_interval => 604800000000 );
f34855556f4d Add timescale creation example.
Mahlon E. Smith <mahlon@laika.com>
parents: 1
diff changeset
    52
```
1
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    53
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    54
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    55
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    56
### Netdata
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    57
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    58
You'll likely want to pare down what netdata is sending.  Here's an
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    59
example configuration for `netdata.conf`:
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    60
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    61
```
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    62
[backend]
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    63
    hostname           = your-hostname
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    64
    enabled            = yes
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    65
    type               = json
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    66
    data source        = average
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    67
    destination        = machine-where-netdata-tsrelay-lives:14866
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    68
    prefix             = n
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    69
    update every       = 10
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    70
    buffer on failures = 6
5
a1276c3d39eb Multiple changes.
Mahlon E. Smith <mahlon@laika.com>
parents: 3
diff changeset
    71
    send charts matching = !cpu.cpu* !ipv6* !users* nfs.rpc net.* net_drops.* net_packets.* !system.interrupts* system.* disk.* disk_space.* disk_ops.* mem.*
1
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    72
```
160338bb2822 Add README.
Mahlon E. Smith <mahlon@laika.com>
parents:
diff changeset
    73