How to Write Points from CSV to InfluxDB v2 and InfluxDB Cloud


Using the Telegraf File Plugin to write CSV to InfluxDB Cloud

Telegraf is InfluxData’s plugin-driven server agent for collecting and reporting metrics. There are over 200 input plugins, which means there are many ways to get data into InfluxDB. However, I frequently see new Influx users inquiring about how to write points from CSV to InfluxDB on the InfluxData Community Site. Writing points from a CSV file is an easy way to insert familiar data into InfluxDB, which can make it easier to get acquainted with the platform.

  • The -sample-config flag will generate the telegraf config file.
  • The -input-filter and -output-filter flags specify the input and output sources of the data, respectively.
  • The text following the > names the config file.
## Files to parse each interval. Accept standard unix glob matching rules,
## as well as ** to match recursive files and directories.
files = ["/path/to/writeCSV.csv"]
## Data format to consume.
## Each data format has its own unique set of configuration options, read
## more about them here:
## Data format to consume.
data_format = "csv"
## Indicates how many rows to treat as a header. By default, the parser assumes
## there is no header and will parse the first row as data. If set to anything more
## than 1, column names will be concatenated with the name listed in the next header row.
## If `csv_column_names` is specified, the column names in header will be overridden.
csv_header_row_count = 1
## Indicates the number of rows to skip before looking for header information.
csv_skip_rows = 1
## Indicates the number of columns to skip before looking for data to parse.
## These columns will be skipped in the header as well.
csv_skip_columns = 1

## The character reserved for marking a row as a comment row
## Commented rows are skipped and not parsed
csv_comment = "#"
## The column to extract the name of the metric from
csv_measurement_column = "_measurement"
## Columns listed here will be added as tags. Any other columns
## will be added as fields.
csv_tag_columns = ["host","region"]
## The column to extract time information for the metric
## `csv_timestamp_format` must be specified if this is used
csv_timestamp_column = "_time"
## The format of time data extracted from `csv_timestamp_column`
## this must be specified if `csv_timestamp_column` is specified
csv_timestamp_format = "2006-01-02T15:04:05Z07:00"
  • Bucket. A bucket is where data is stored. You can think of it as a database. All buckets have retention policies. You assign a bucket a retention policy upon bucket creation. The easiest way to create a bucket is through the InfluxDB Cloud user interface (UI), as described here. However, you can also use the CLI or API to create a bucket. This bucket will be referred to as our destination bucket, “mybucket”.
  • Token: An InfluxDB authentication token ensures security for users and their data. Tokens belong to organizations and describe permissions. You’ll need to create a read/write or all-access token scoped to your destination bucket. The easiest way to create a bucket is through the InfluxDB Cloud UI, as described here. However, you can also use the CLI or API to create a token.
  • Organization: Your organization will be the email you used to set up your InfluxDB Cloud account.
# Configuration for sending metrics to InfluxDB 2.0
## The URLs of the InfluxDB cluster nodes.
## Multiple URLs can be specified for a single cluster, only ONE of the
## urls will be written to each interval.
## ex: urls = [""] if your using OSS
urls = [""]

## Token for authentication.
token = "mytoken"

## Organization is the name of the organization you wish to write to.
organization = ""

## Destination bucket to write into.
bucket = "mybucket"

Using the CLI to write CSV to InfluxDB Cloud

First you must download the CLI in order to write a CSV to InfluxDB Cloud with the CLI. Use the following command to write a CSV to your InfluxDB destination bucket:

influx write -b mybucket\
-t mytoken
-f path/to/writeCSV.csv \
--header "#datatype dateTime:2006-01-02T15:04:05Z07:00,tag,tag,double,measurement”
influx write dryrun  -b mybucket -f ~/path/to/writeCSV.csv --header "#datatype dateTime:2006-01-02T15:04:05Z07:00,tag,tag,double,measurement"
cpu,host=A,region=east usage_user=15.43 1525812600000000000
cpu,host=A,region=east usage_user=59.25 1525812620000000000
cpu,host=A,region=east usage_user=52.62 1525812640000000000

Using the csv.from() Flux function to write CSV to InfluxDB Cloud

The data scripting language for InfluxDB is called Flux. Flux enables you to…

import "csv"
csvData = "
csv.from(csv: csvData, mode: "raw")
|> rename(columns: {usage_user: "_value"})
|> map(fn: (r) => ({ r with _value: float(v: r._value) }))
|> map(fn: (r) => ({ r with _time: time(v: r._time) }))
|> range(start: v.timeRangeStart, stop: v.timeRangeStop)
|> to(bucket: "mybucket")
  • Store your CSV data in a variable.
  • Specify the path to your CSV. Please note that the CSV must exist in the same file system running the fluxd process. In other words, this approach doesn’t work in the InfluxDB UI for InfluxDB Cloud.
  • Then we use the rename() function to rename the usage_user column to “_value”. This column is renamed because the InfluxDB UI visualizes columns with the name “_value” by default.
  • The csv.from() function will convert our data to strings by default.
  • The map() function applies the time() function to each record in the “_time” column.
  • The time() function converts the string value to a time.

Using the Python client to write CSV to InfluxDB Cloud

The InfluxDB Python Client supports Pandas and pandas dataframes which I believe is the easiest way to write the CSV to InfluxDB. Your code would look like this:

import pandas as pd
from influxdb_client import InfluxDBClient

client = InfluxDBClient(url="", token="mytoken", org="myorg")
write_client = client.write_api()

df = pd.read_csv("~/Desktop/writeCSV.csv")
df['_time'] = pd.to_datetime(df['_time'], format="%Y-%m-%dT%H:%M:%SZ")

write_client.write("mybucket", "", record=df, data_frame_measurement_name="_measurement",
data_frame_tag_columns=['region', 'host'])

Conclusions about writing CSV to InfluxDB v2 Cloud and OSS

I hope this InfluxDB blog post helps you to get started using InfluxDB and writing CSV to InfluxDB. If you are using the new InfluxDB and need help, please ask for some in our community site or Slack channel. If you’re developing a cool IoT application or monitoring your application on top of InfluxDB, we’d love to hear about it, so make sure to share your story! Additionally, please share your thoughts, concerns, or questions in the comments section. We’d love to get your feedback and help you with any problems you run into!

Further reading

While this post aims to provide a comprehensive overview of how to write CSV to InfluxDB, the following resources might also interest you:

  1. TL;DR InfluxDB Tech Tips — How to Use the CLI Locally to Access the Flux REPL and Write a Regular CSV to InfluxDB Cloud: This post shares how to use the InfluxDB CLI with InfluxDB Cloud to write a CSV to InfluxDB.
  2. TL;DR InfluxDB Tech Tips — How to Interpret an Annotated CSV: This post describes how to interpret an Annotated CSV, the Flux query result format for InfluxDB.
  3. Getting Started with Python and InfluxDB v2.0: This post describes how to get started with the Python Client Library.
  4. Getting Started with InfluxDB and Pandas: This post shares how to get started with Pandas to write dataframes to InfluxDB and return dataframes with a Flux query.
  5. Top 5 Hurdles for Flux Beginners and Resources for Learning to Use Flux: This post describes common hurdles for Flux beginners and how to tackle them by using the InfluxDB UI, understanding Annotated CSV, and more.
  6. Top 5 Hurdles for Intermediate Flux Users and Resources for Optimizing Flux: This post describes common hurdles for intermediate and advanced Flux users while providing more detail on pushdown patterns, how the Flux engine works, and more.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Anais Dotis

Anais Dotis

Developer Advocate at InfluxData