Note how the config looks like.
We are looking for a solution to display snowflake to Grafana. I know that have a schedule to pull the data out of the data lake looks strange, but with no choice. Unfortunately, using Grafana to pull data directly from snowflake is expensive. Therefore we need an alternative. Below is the data flow:
snowflake ← logstash → ES
Here is the logstash JDBC input example
logstash.yml with snowflake JDBC connector
input {
jdbc {
jdbc_driver_library => "/etc/logstash/pipeline/snowflake/dependency/snowflake-jdbc-3.9.2.jar"
jdbc_driver_class => "com.snowflake.client.jdbc.SnowflakeDriver"
jdbc_connection_string => "jdbc:snowflake://url.eu-west-1.snowflakecomputing.com/?db=dbname&warehouse=House"
jdbc_user => "user"
jdbc_password => "password"
schedule => "15 * * * *"
statement => "
SELECT TOP 100000 * FROM dbname.CORE.table WHERE timestamp >= '2038-10-02 00:00:00' AND timestamp < '2038-10-02 23:59:59'
"
jdbc_fetch_size => 1000
}
}