This project provides SolarNode plugin that can collect data from URLs returning comma-separated-value (CSV) data, either as a node or location datum stream.

The plugin can be installed via the Plugins page on your SolarNode. It appears under the Datum category as CSV Datum Source.

Once installed, new CSV Resource and CSV Location Resource components will appear on the Settings page on your SolarNode. Click on the Manage button to configure components. You'll need to add one configuration for each CSV lookup you want to collect data from.

All CSV column reference settings accept a list of column ranges. When more than one column is configured, then all column values are joined together with a (space) character to form a single final value.

A column can be referenced as a number, starting from 1, or a letter, starting from A (common in spreadsheet applications). After Z comes AA, AB, AC, and so on.

A range of columns can be referenced using a - (dash) delimiter. For example 1-3 means columns 1, 2, and 3 will be included. Similarly A-C would include the same columns. Letters and numbers can be freely mixed. For example 1-C would include the same columns. The letters are also case-insensitive, so a-c would include the same columns.

Multiple columns, or ranges of columns, can be referenced using a , (comma) delimiter. For example 1,3 means columns 1 and 3 would be included. Similarly A,C would include the same columns. A more complex example of 1,F-H,3 would include columns 1, 6, 7, 8, and 3. The order of the references are preserved when joining the columns together with a (space) character.

This plugin also provides CSV Resource CSV Configurer and CSV Location Resource CSV Configurer components; these will appear on the main settings page. These components let you upload a CSV Configuration file to configure all CSV [Location] Resource components, without having to use the settings form.

The CSV Resource component collects CSV data as a node datum stream. See CSV Location Resource if you need to collect a location datum stream.

The CSV Resource CSV Configurer uses the column structure detailed below, with each row representing an individual datum property to read from the CSV data. A header row is required. Comment lines are allowed, just start the line with a # character (i.e. the first cell value). The entire comment line will be ignored.

Here's an example screen shot of a configuration in a spreadsheet application. It is for two components:

1. Resource Carpark with 3 datum properties: occupied, evFree, and evChargePower
2. Resource Airport with 2 datum properties: watts and wattHours

Spreadsheet applications generally allow you to export sheets in the CSV format, which can then be loaded into SolarNode via the CSV Resource CSV Configurer.

Individual CSV Resource components are defined by the first column (Instance ID). You can assign any identifier you like (such as Meter, Inverter, and so on) or configure as a single dash character - to have SolarNode assign a simple number identifier. Once an Instance ID has been assigned on a given row, subsequent rows will use that value if the corresponding cell value is left empty.

Here's an example of how 3 custom instance IDs Meter, Pyranometer, and Weather appear in the SolarNode UI, with the Pyranometer component selected:

The following table defines all the CSV columns used by CSV Resource CSV Configuration. Columns A - Q apply to the entire CSV Resource configuration, and only the values from the row that defines a new Instance ID will be used to configure the component. Thus you can omit the values from these columns when defining more than one property for a given component.

Columns S - W define the mapping of CSV columns to datum properties: each row defines an individual datum property.

Here is the CSV as shown in the example configuration screen shot above (comments have been removed for brevity):

Instance ID,Service Name,Service Group,Source ID,Source ID Column,Schedule,URL,Customizer,Encoding,Timeout,Skip,Keep,URL Date Format,Date Column,Date Format,Time Zone,Sample Cache,Column,Property,Property Type,Multiplier,Offset,Decimal Scale
Carpark,Garage,,garage/1,,0 * * * * *,https://example.com/garage?date={date},,UTF-8,15000,-1,1,yyyy-MM-dd'T'HH:mm'Z',A,yyyy-MM-dd'T'HH:mm'Z',UTC,55000,B,occupied,Instantaneous,1,0,0
,,,,,,,,,,,,,,,,,D,evFree,Instantaneous,1,0,0
,,,,,,,,,,,,,,,,,G,evChargePower,Instantaneous,1000,0,0
Airport,Hangar,,,A-C,0 * * * * *,https://example.com/airport?date={date},,US-ASCII,30000,1,1,yyyy-MM-dd,D-E,dd/MM/yyyy HH:mm,Pacific/Auckland,55000,C,watts,Instantaneous,1,0,0
,,,,,,,,,,,,,,,,,E,wattHours,Accumulating,1,0,0

Each CSV Resource configuration contains the following settings:

You must configure settings for each datum property you want to collect from each component. You can configure as many property settings as you like, using the + and - buttons to add/remove property configurations.

Each CSV Resource property configuration contains the following settings:

The CSV Location Resource component collects CSV data as a location datum stream. See CSV Resource if you need to collect a node datum stream.

The CSV Location Resource Configurer is similar to the CSV Resource Configurer:

The CSV Location Resource CSV column definition is similar to the CSV Resource CSV column definition. Only the following columns are different:

Each configuration is very similar to the CSV Resource Settings. There are no Source ID or Source ID Column settings, because those are provided by the configured Location. The following additional settings are available:

The Location and Location Type settings are used to associate the data with a SolarNetwork location datum stream. These locations must be created by SolarNetwork administrators. The Location setting will allow you to search for available locations. You can also use the Find location datum metadata API to search for locations. You can use the Location Request API to request a new location if a suitable one does not exist already.