New episodes every weekday Monday through Friday. This page was generated by The HPR Robot at
Welcome to HPR, the Community Podcast
We started producing shows as Today with a Techie on 2005-09-19, 19 years, 0 months, 26 days ago. Our shows are produced by listeners like you and can be on any topics that "are of interest to hackers". If you listen to HPR then please consider contributing one show a year. If you record your show now it could be released in 1 days.
Call for shows
We are running very low on shows at the moment. Have a look at the hosts page and if you don't see "2024-??-??" next to your name, or if your name is not listed, you might consider sending us in something.
In this episode we will continue looking at basic filters. Then we
will start looking at the feature that makes jq very
powerful, the ability to transform JSON from one form to another. In
essence we can read and parse JSON and then construct an alternative
form.
More basic filters
Array/String Slice:
.[<number>:<number>]
This filter allows parts of JSON arrays or strings to be
extracted.
The first number is the index of the elements of the array or string,
starting from zero. The second number is an ending index, but it means
"up to but not including". If the first index is omitted it refers to
the start of the string or array. If the second index is blank it refers
to the end of the string or array.
This example shows using an array and extracting part of it:
Here we use the seq command to generate the numbers 1-10
separated by commas in a JSON array. Feeding this to jq on
its standard input with the slice request '.[3:6]' results
in a sub-array from element 3 (containing value 4), up to but not
including element 6 (containing 7). Note that using the
'-c' option generates compact output, as we discussed in
the last episode.
For a string, the idea is similar, as in:
$ echo '"Hacker Public Radio"' | jq '.[7:10]'
"Pub"
Notice that we provide the JSON string quotes inside single quotes
following echo. The filter '.[7:10]' starts
from element 7 (letter "P") up to but not including element 10
(letter "l").
Both of the numbers may be negative, meaning that they are offsets
from the end of the array or string.
So, using '.[-7:-4]' in the array example gives the same
result as '.[3:6]', as do '.[3:-4]' and
'.[-7:6]'. This example uses the x variable
created earlier:
$ for f in '.[-7:-4]' '.[3:6]' '.[3:-4]' '.[-7:6]'; do
> echo "$x" | jq -c "$f"
> done
[4,5,6]
[4,5,6]
[4,5,6]
[4,5,6]
Similarly, using '.[-12:-9]' gives the same result as
'.[7:10]' when used with the string.
$ echo '"Hacker Public Radio"' | jq '.[-12:-9]'
"Pub"
As a point of interest, I wrote a little Bash loop to show the
positive and negative offsets of the characters in the test string -
just to help me visualise them. See the footnote1 for details.
Finally, here is how to get the last character of the example string
using positive and negative offsets:
$ echo '"Hacker Public Radio"' | jq '.[18:]'
"o"
$ echo '"Hacker Public Radio"' | jq '.[-1:]'
"o"
Array/Object Value Iterator:
.[]
This filter generates values from iterating through an array or an
object. It is similar to the .[index] syntax we have
already seen, but it returns all of the array elements:
There are two operators that can be placed between filters to combine
their effects: the comma (',') and the
pipe ('|').
Comma operator
The comma (',') operator allows you to chain together
multiple filters. As we already know, the jq program feeds
the input it receives on standard input or from a file into whatever
filter it is given. So far we have only seen a single filter being
used.
With the comma operator the input to jq is fed to all of
the filters separated by commas in left to right order. The result is a
concatenation of the output of all of these filters.
For example, if we take the output from the HPR stats page which was
mentioned in part 1
of this series of shows, and store it in a file called
stats.json we can view two separate parts of the JSON like
this:
This applies the filter .shows (an object
identifier-index filter, see part 2)
which returns the contents of the object with that name, then it applies
filter .queue which returns the relevant JSON object.
Pipe operator
The pipe ('|') operator combines filters by feeding the
output of the first (left-most) filter of a pair into the second
(right-most) filter of a pair. This is analogous to the way the same
symbol works in the Unix shell.
For example, if we extract the 'shows' object from
stats.json, we can then extract the value of the
total' key' as follows:
$ jq '.shows | .total' stats.json
4756
Interestingly, chaining two object identifier-index filters
gives the same output:
$ jq '.shows.total' stats.json
4756
(Note: to answer the question in the audio, the two filters shown can
also be written as '.shows .total'with intervening
spaces.)
We will see the pipe operator being used in many instances in
upcoming episodes.
Parentheses
It is possible to use parentheses in filter expressions in a similar
way to using them in arithmetic, where they group parts together and can
change the normal order of operations. They can be used in other
contexts too. The example is a simple arithmetic one:
Here we are using a file called countries.json obtained
from the GitHub
project listed below. This file is around 39,000 lines long so it is
not being distributed with the show. However, it's quite interesting and
you are encouraged to grab a copy and experiment with it.
I will show ways in which the structure can be examined and reported
with jq in a later show, but for now I will show an example
of extracting data:
The file contains an array of country objects; the one with index 42
is Switzerland.
The name of the country is in an object called "name",
with the common name in a keyed field called "common", thus
the filter .name.common.
In this country object is an object called "capital"
holding an array containing the name (or names) of the capital city (or
cities). The filter .capital.[] obtains and displays the
contents of the array.
Note that we used a comma operator between the
filters.
Finding country data #2
Another search of the countries.json file, this time
looking at the languages spoken. There is an object called
"languages" which contains abbreviated language names as
keys and full names as the values:
PREP TIME
5 minutes
TOTAL TIME
5 minutes
Ingredients
½ cup mayonnaise
½ cup sour cream
3 Tablespoons pickled jalapeno juice (from a jar of pickled jalapenos)
3 Tablespoons pickled jalapenos (diced)
2 teaspoon paprika
2 teaspoons ground cumin
1 teaspoon garlic granules
1 teaspoon onion powder
½ teaspoon salt (or to taste)
½ teaspoon chili powder
Known in the Hacker world as the Ham Radio Guy at FOSDEM,
Kristoff Bonne ON1ARF, and
Ken Fallon
PA7KEN/G5KEN sit down to
discuss
"vergrijzing,
changing attitudes, and increasing participation in the Amateur Radio
Community."
The Artifice Girl is a 2022 science fiction psychological thriller
written and directed by Franklin Ritch, produced by Aaron B. Koontz and
released direct to VOD. It stars Tatum Matthews, Sinda Nichols, David
Girard, Lance Henriksen, and Franklin Ritch. NGO agents discover a
revolutionary Artificial intelligence (AI) computer program that uses a
digital child to catch online predators, it advances far more rapid than
they could have imagined, posing unforeseen challenges for the
relationship between humans and AI.
Shows the EPROM (Erasable Programmable Read Only Memory) fitted with a
label marked JAF7. EPROM's can be erased by removing the sticker and
exposing the device to strong ultraviolet light. I incorrectly refer to
it as a PROM (Programmable Read Only Memory. Below it is the old leaking
PLL backup battery.
Link
/ example of a Fluke 77 DMM (Digital Multi Meter) I mention that I
used it to check the battery voltage of the original PLL backup battery
which I think was manufactured around 1984. I was very surprised to find
that the battery which is likely 40 years old had a battery voltage of
3.2V which is a healthy voltage. The battery still needed changed due to
liquid (probably acid) which can be seen in picture 1 being present on
the top surface of the battery.
Picture 2
Shows me extending the new battery terminal using the leg of a 1.8 kilo
ohm resistor. Note that the resistor leg is shiny because this time I
scraped away the surface oxidation using pliers prior to soldering it in
place. This improves the final solder joint.
These are comments which have been made during the past month, either to shows released during the month or to past shows.
There are 23 comments in total.
Policy decisions surrounding HPR are taken by the community as a whole. This
discussion takes place on the Mail List which is open to all HPR listeners and
contributors. The discussions are open and available on the HPR server under
Mailman.
The threaded discussions this month can be found here:
This is the LWN.net community event calendar, where we track
events of interest to people using and developing Linux and free software.
Clicking on individual events will take you to the appropriate web
page.
Any other business
Repairing
shows where external files have been lost
The further back in time we go with these repairs, the more
challenging they tend to become because of the variations in the way
shows were put together. It has been difficult to process more than five
a day, and there have been a few breaks along the way!
The current state of the project is that it has been
completed:
All of the processed shows have access to their transcripts, which
are held on archive.org. However, there remains the need to make
transcripts available to the older shows which have no external
files.
This is a look at the very first serial of Doctor who, and how the
cast was first put in place. It is in many ways an unlikely story for a
show that has gone on for 60 years and counting. I find the early
stories to be charming and worth a look, and many things from these
early stories come back in later stories as well.
Dual-fuel design allows you to power the headlamp with 3 AAA
alkaline batteries (included) or the rechargeable BD 1500 lithium-ion
battery and charger (not included)
Settings include proximity and distance modes, dimming, strobe, red
LED night vision, and lock mode
White LED outputs up to 400 lumens on max setting, 200 lumens on
medium and 6 lumens on low
PowerTap™ Technology allows instant adjustment between max output
and dimmed power
Brightness Memory allows you to turn the light on and off at a
chosen brightness setting without it reverting back to the default
setting
Red LED night vision has dimming and strobe modes, and is quickly
activated without cycling through the white mode
Multifaceted optical efficiency lens technology
Integrated battery meter display shows the percentage of power
remaining so you know when to replace the batteries
Digital lockout feature prevents accidental use when stored in a
pack or pocket
IPX8 waterproof protection rated to operate at least 1.1 meters
underwater for 30 minutes—perfect should you be caught in a
downpour
If submerged, water may enter the battery compartment, but it will
still operate; it should be completely dried out after use in wet
conditions