Site Map - skip to main content

Hacker Public Radio

Your ideas, projects, opinions - podcasted.

New episodes every weekday Monday through Friday.
This page was generated by The HPR Robot at


hpr3023 :: Critique My Script, Episode 1 - Qots-Crew-Gen

Discussion of using a shell script to randomly generate a ten man aircrew.

<< First, < Previous, , Latest >>

Hosted by Carl on Wednesday, 2020-03-04 is flagged as Clean and is released under a CC-BY-SA license.
Shell Script, Random Numbers, Awk. 11.
The show is available on the Internet Archive at: https://archive.org/details/hpr3023

Listen in ogg, spx, or mp3 format. Play now:

Duration: 00:13:02

general.

This is my second HPR episode and the first in what could be a series about shell scripts I have written. This episode goes through a short script which randomly generates first and last names for a ten man aircrew to use with the Avalon Hill game B-17 Queen of the Skies.

You can see the basic script in action here:
https://www.sodface.com/misc/qots-crew-gen

and a more complicated version here, though based on the same underlying methodolgy:
https://www.sodface.com/misc/qots-crew-gen2

Here’s the script:

#!/bin/sh

first_names='./firstnames.txt'
last_names='./surnames.txt'
crew_positions='./positions.txt'
crew_ranks='./ranks.txt'

len_first_names=$(wc -l < ${first_names})
len_last_names=$(wc -l < ${last_names})

num_pairs=$(printf "10 ${len_first_names} ${len_last_names}" | \
    awk 'BEGIN { srand() }
    { for (i=1; i<=$1; i++) {
        for(f = 2; f <= NF; f++) {
        num=int(rand() * $f + 1); printf num"," } printf "\n"
    }
    }')

i=1

for crew_member in ${num_pairs}
do
  line_num=$(printf "${crew_member}" | cut -d',' -f1)
  first_name=$(sed -n ${line_num}p ${first_names})

  line_num=$(printf "${crew_member}" | cut -d',' -f2)
  last_name=$(sed -n ${line_num}p ${last_names})

  crew_position=$(sed -n ${i}p ${crew_positions})
  crew_rank=$(sed -n ${i}p ${crew_ranks})

  #
  # Use the variables above to generate HTML.
  # Omitted here to simplify this example.
  #

  i=$((( ${i} +1 )))
done

Comments

Subscribe to the comments RSS feed.

Comment #1 posted on 2020-03-04 10:49:34 by Dave Morriss

Bash arithmetic

The expression you use to increment 'i' stands out to me:

i=$((( ${i} +1 )))

Bash has pre- and post-increment arithmetic expressions and there's a compound command which lets you use:

((i++))

Look for "compound commands" and "arithmetic evaluation" in the Bash documentation. I covered some of this in https://hackerpublicradio.org/eps.php?id=1951

For example, the following one-liner sets and increments 'i' as you do:

i=1; for name in A B C; do echo "$i: $name"; ((i++)); done
1: A
2: B
3: C

Comment #2 posted on 2020-03-04 16:08:55 by Dave Morriss

Another Bash-ism that might be useful

I appreciate that you are not using Bash in your script, but unless you have some strong reason not to I'd advise using it. Often 'sh' is just a restricted form of Bash!

If you agree then you can change things like:

line_num=$(printf "${crew_member}" | cut -d',' -f1)
line_num=$(printf "${crew_member}" | cut -d',' -f2)

into:

line_num="${crew_member%,*}" # gets first element
line_num="${crew_member#*,}" # gets second element

This only deals with two-element comma-separated lists so it's not quite as flexible as 'cut'.

The % provides suffix removal and # prefix removal. I covered this in show 1648 (https://hackerpublicradio.org/eps.php?id=1648)

Comment #3 posted on 2020-03-04 18:08:21 by nobody

There must be an easier way

The biggest problem with your scripting seems to be that you don't really know what tools are available and what options they have. My recommendation would be for you to just see what comes included with the coreutils and busybox, you'll find all kinds of wonderful little tools there.

In bash my solution to this would be this single line:

paste ./ranks.txt ./position.txt <(shuf -n 10 first.txt) <(shuf -n 10 last.txt)

If you don't mind two crewmen having two the same name then you can add -r to the shuf flags, at least when using GNU coreutils.

Since busybox' ash lacks the wonderful process substitution of bash (the <(cmd) in the above) I would probably just do something like this:

while read -r x do
printf '%s %s %sn' "$x" "$(shuf -n 1 first.txt)" "$(shuf -n 1 last.txt)"
done <(paste ranks.txt position.txt)

With larger files this ash compatible version would be quite inefficient and slow but I doubt that really matters here.

Comment #4 posted on 2020-03-04 18:11:54 by nobody

Little correction to my comment

Forgot to actually remove the bashism... and forgot a semicolon

paste ranks.txt position.txt | while read -r x; do
printf '%s %s %sn' "$x" "$(shuf -n 1 first.txt)" "$(shuf -n 1 last.txt)"
done

Comment #5 posted on 2020-03-05 04:04:22 by Carl

Thanks for the comments

I asked for a critque so I appreciate the comments!

@Dave
I don't have any _strong_ reasons for not using bash, but it boils down to:
- Most of the limited scripting I do is on Raspberry Pi and other SBC type devices, usually with Alpine Linux, which out of the box has /bin/sh as a symlink to busybox, so I work with that in lieu of installing bash.

- I sort of like the extra challenge of not using bashisms, even if it does make things a bit harder/uglier than it needs to be.

I'm at a loss to explain where I came up with the triple parentheses for incrementing i. I just tried it on busybox and two seems to work fine (though three does also). The ++ form does not (as you note it would require bash) though I'm familiar with that form, it just doesn't work within the constraints of busybox.

Comment #6 posted on 2020-03-05 04:05:12 by Carl

Thanks for the comments

@nobody

I looked at shuf for this but it's not a busybox builtin and not included out of the box with Alpine, though awk is, which is why I went with it to generate the random number pairs. So to say that I don't know what tools are available is perhaps a little unfair as I did state in the episode that I'm limiting myself to busybox builtins. Imposing that limitation on myself is perhaps a little silly, but, it is a fact that I would have to install _something_ to get the additional functionality you reference, and that may not always be possible or desirable in embedded applications.

Actually, I wrote the above so I'll leave it there, but I decided to double check. I usually refer to https://busybox.net/BusyBox.html as a single page reference to the builtins and shuf isn't listed but /usr/bin/shuf is indeed a symlink to /bin/busybox on one of my Alpine devices, which is a little annoying. When I work on a script like this one, I usually do it on my laptop that has all the full tools on it but I double check against the busybox page to make sure I'm not using a command or an option to a command that busybox doesn't support. Then I test it on one of the devices.

Comment #7 posted on 2020-03-05 10:03:34 by nobody

Re: Re:

To see what tools your Busybox come with you should run it without options. Busybox is quite configurable so you should check documentation generated with the same configuration as your target. That web page is either very outdated or generated from some sample configuration.

Besides, ash also has $RANDOM so using AWK isn't really necessary:

echo $((RANDOM%firstnames_len))

Here is also a fix for Dave's suggestion:

i=1; for name in A B C; do echo "$((i++)): $name"; done

And if you use preincrement the i=1 is also unnecessary:

for name in A B C; do echo "$((++i)): $name"; done

Comment #8 posted on 2020-03-05 12:02:25 by nobody

Standalone increment in ash

Also if you want the ((i++)) increment for ash you could pretty easily replicate it with:

: $((i++))

Comment #9 posted on 2020-03-05 12:32:11 by Carl

Neat

> echo $((RANDOM%firstnames_len))
That's neat. The first time I tried it though I got the same number twice in a row:
m300-01:/srv$ echo $((RANDOM%100))
88
m300-01:/srv$ echo $((RANDOM%100))
88
m300-01:/srv$ echo $((RANDOM%100))
68

Probably just a fluke but I'd be interested to test it in rapid succession (eg. a loop) to see if it's more prone to do that than the awk random number generator.

To be clear, you're not suggesting the pre/post increments work on busybox/ash correct - they don't appear to unless I'm doing it wrong:

-ash: arithmetic syntax error

Comment #10 posted on 2020-03-05 16:30:40 by nobody

$(())

>to see if it's more prone to do that than the awk random number generator
I doubt there is any significant difference. Certainly not any that would matter for a project like this.

>To be clear, you're not suggesting the pre/post increments work on busybox/ash correct - they don't appear to unless I'm doing it wrong:
I just threw up a Alpine container and at least there it works just fine. What command did you run that produced this error message?

Comment #11 posted on 2020-03-05 21:11:58 by Carl

Version 3

@nobody, not sure what I did earlier to produce that arithmetic error, I just tried it again and your examples are working. Sorry about that.

I just did a third version:
https://www.sodface.com/misc/qots-crew-gen3

Greatly simplified, no loops and just using shuf repeatedly per nobody's example to get the first and last name.

https://pastebin.com/iaXw9ZL2

Thanks to both Dave and nobody for the feedback.

Leave Comment

Note to Verbose Commenters
If you can't fit everything you want to say in the comment below then you really should record a response show instead.

Note to Spammers
All comments are moderated. All links are checked by humans. We strip out all html. Feel free to record a show about yourself, or your industry, or any other topic we may find interesting. We also check shows for spam :).

Provide feedback
Your Name/Handle:
Title:
Comment:
Anti Spam Question: What does the letter P in HPR stand for?
Are you a spammer?
Who is the host of this show?
What does HPR mean to you?