A couple lines / documentation on how gsheet takes stdin could be extremely helpful. Maybe even just a few more examples of how it can be done. I’ll keep working on it, and if I develop some facility on how the software works, I’ll throw in. Just not there yet. Details below.
—-
When piping data into gsheet csv, I ran into issues getting the program to recognize separate CSV objects. My CSV data is coming into gsheet on separate lines and I struggled to get the utility to recognize each line as a new row of data. I have a slow data source that I’m sending through jq for filtering and generating CSV before sending along to gsheet. I went down a rabbit hole on buffer control for piping. I ended up needing a helper script to take each line and feed it to gsheet as a separate instance, otherwise, it seemed to keep waiting for the data stream to close before ingesting, processing, and uploading the data. I wanted to do it continuously on a persistently open data source.
slowDataJSON | jq -r --unbuffered '[.[]]|@csv' | stdbuf --output=L | tr -d '"' | /home/timmy/parseEachLine.sh
Data coming into the helper script below as:
Foo, Bar, Baz
Bar, Baz, Foo
Baz, Foo, Bar
My desire for each line to be read in as a row of data in the sheet.
#!/bin/bash
#parseEachLine.sh
while IFS=$'\n' read -r line; do
echo $line | /home/timmy/go/bin/gsheet csv --append --id=$SHEET_ID --range='Sheet1!'A:B --append
done
This solution works, but it’s not concise or efficient. I definitely took the long trip to get to my destination; I’m sure there’s a right way that would have been much simpler. Some documentation on the right way to do it would be helpful.
(Awesome work from the team. Very handy utility.)
A couple lines / documentation on how gsheet takes stdin could be extremely helpful. Maybe even just a few more examples of how it can be done. I’ll keep working on it, and if I develop some facility on how the software works, I’ll throw in. Just not there yet. Details below.
—-
When piping data into gsheet csv, I ran into issues getting the program to recognize separate CSV objects. My CSV data is coming into gsheet on separate lines and I struggled to get the utility to recognize each line as a new row of data. I have a slow data source that I’m sending through jq for filtering and generating CSV before sending along to gsheet. I went down a rabbit hole on buffer control for piping. I ended up needing a helper script to take each line and feed it to gsheet as a separate instance, otherwise, it seemed to keep waiting for the data stream to close before ingesting, processing, and uploading the data. I wanted to do it continuously on a persistently open data source.
Data coming into the helper script below as:
My desire for each line to be read in as a row of data in the sheet.
This solution works, but it’s not concise or efficient. I definitely took the long trip to get to my destination; I’m sure there’s a right way that would have been much simpler. Some documentation on the right way to do it would be helpful.
(Awesome work from the team. Very handy utility.)