From 1cdd26fc76b46372dd467511b7ad51d790ffde0d Mon Sep 17 00:00:00 2001 From: Lance Math Date: Thu, 20 Jan 2022 01:11:17 -0800 Subject: [PATCH 1/2] added hints to a1 --- labs/a1.md | 151 +++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 119 insertions(+), 32 deletions(-) diff --git a/labs/a1.md b/labs/a1.md index b5f0568a..af3fb2d3 100644 --- a/labs/a1.md +++ b/labs/a1.md @@ -37,50 +37,87 @@ or anything of the sort to worry about (which will also be true for the followin We will release sample solutions after the lab is due, but keep in mind that there are many ways to solve these problems. +## Shell Scripting Review +A quick refresher on writing shell scripts. Feel free to skip anything that looks familiar. -# Question 1 +### Shebang! -At some point, everyone has looked at a problem and thought to themselves: -"Hey, I can do this in one line!" +Shell scripts typically begin with the shebang line: #!path/to/interpreter. -Lets find out if you can. I need to sort out some of my most listened to albums -by making directories for each of them, specifically for my favorite artist Future. +`#!` is a human-readable representation of a [magic number][1] `0x23 0x21` +which can tell the shell to pass execution of the rest of the file to a +specified interpreter. If your script is run as an executable (e.g. +`./awesome_shell_script`) with a shebang line, then the shell will invoke the +executable (usually an interpreter) at `path/to/interpreter` to run your +script. If your script is passed as an argument to an interpreter e.g. `bash +awesome_shell_script`, then the shebang has no effect and `bash` will handle +the script's execution. -I have hosted a list of my favorite albums followed by their respective artist at -`https://raw.githubusercontent.com/0xcf/decal-labs/master/a1/albums.txt`, -in a comma delimited format like `Die Lit, Playboi Carti`. For the GOAT artist Future, -I want to create a folder for each of his albums. For example, for an entry like -`SUPER SLIMEY, Future` I would expect a directory called `SUPER SLIMEY` to be created. +**Why is this important?** The shebang line can be considered a useful piece of +metadata which passes the concern of _how_ a script is executed from the user +to the program's author. `awesome_shell_script` could be a `bash` script, a +`python` script, a `ruby` script, etc. The idea is that only the script's +behavior, not its implementation details, should matter to the user who calls +the script. -**TLDR**: You need to fetch the list from the web, filter out the albums we want, -trim out the album name, and then make a directory for each one, all in one line. +You may have seen some variant of `#!/bin/sh`. Although initially referencing +the Bourne shell, on modern systems `sh` has come to reference to the [Shell +Command Language][2], which is a POSIX specification with many implementations. +`sh` is usually symlinked to one of these POSIX-compliant shells which +implement the Shell Command Language. On Debian, for instance, `sh` is +symlinked to the shell `dash`. It is important to note that `bash` does **not** +comply with this standard, although running `bash` as `bash --posix` makes it +more compliant. + +### Piping + +We can use the `|` character to chain multiple commands together in one line. For instance: `command1 | command2` +will pass the output of `command1` as the input to `command2`. We can repeat this as many times as we need. + +### Looping with `for` + +Bash can repeat an operation on multiple objects with a for loop. The syntax is as follows: ```bash -$ cat albums.txt -... -Drip or Drown 2, Gunna -Playboi Carti, Playboi Carti -DS2 (Deluxe), Future <-- GOAT album detected! -Drip Harder, Lil Baby -The WIZRD, Future <-- GOAT album detected! -What a Time To Be Alive, Drake -... +for VARIABLE in LIST; do -# After our magic one liner... + CONSEQUENT-COMMANDS -$ ls -'DS2 (Deluxe)' 'The WIZRD' ... -# We got our new directories! +done ``` -**Hints:** -- What common text manipulation commands can help you solve this? -- As always, be aware that there isn't one unique solution to this problem! -- Also be aware that `xargs` behaves differently on different platforms. +Indentation is not required, but makes the code easier to read. +The `LIST` can either be something like a directory with multiple files or a file with multiple lines init, a list of files (`file1 file2 file3`), or even a range of numbers (`{start..end}`). -Submit your one line solution on Gradescope! +## Useful Commands +Some commands which could be useful in completing the lab. Of course, there are many approaches to the problems, and using these commands is not required. -# Question 2 +### `cut` + +`cut [options] [filename]` extracts certain parts of a file (or piped-in input) based on the [arguments][3] that are used. A few which might be useful: + +`-d` allows us to change the delimeter, or change the character `cut` looks for to divide the string into chunks. If this option is omitted, `tab` is used. + +`-f` allows us to specify a number corresponding to the field to return, e.g. `cut -f1 -d" "` would return the first word in a sentence. The number followed by a `-` returns all fields after the specified field as well, so `cut -f1- -d" "` would return the entire string. + +`--complement` tells `cut` to reutrn evertything **except** for the specified field. + +### `grep` + +`grep [pattern] [filename]` [filters out][4] and returns lines from a file (or piped-in input) that contain the specified pattern. + +### `xargs` + +`xargs` lets us apply a command to an output redirected from a pipe. For instance, `output | xargs command` would apply `command` to `output`. Some useful [options][5]: + +`-n1` tells `xargs` to apply the command to every item in `output` once if output has multiple items in it (such as a list of multiple strings) + +`-0` tells `xargs` to split items in the output by the null character, which signifies the end of a string, instead of using spaces. Paired with `-n1`, this means `xargs` will apply a command to every string, instead of breaking up strings into individual words and applying the command to every word. + +As always, there are many more ways to use these commands, so use Google or [man pages][6] to learn more! + + +# Question 1 I like Lisp and Scheme, and miss `car` and `cdr` in my usual programming tasks.[^carcdr] @@ -118,7 +155,7 @@ ab/abizer/some/path [carcdrarticle]: https://medium.com/@aleksandrasays/my-other-car-is-a-cdr-3058e6743c15 -# Question 3 +# Question 2 With the invention of the `.norm` file format, file extension innovation is at its peak![^xkcd] @@ -165,6 +202,47 @@ use [shell parameter expansion][1]. [1]: https://stackoverflow.com/questions/965053/extract-filename-and-extension-in-bash +# Question 3 + +At some point, everyone has looked at a problem and thought to themselves: +"Hey, I can do this in one line!" + +Lets find out if you can. I need to sort out some of my most listened to albums +by making directories for each of them, specifically for my favorite artist Future. + +I have hosted a list of my favorite albums followed by their respective artist at +`https://raw.githubusercontent.com/0xcf/decal-labs/master/a1/albums.txt`, +in a comma delimited format like `Die Lit, Playboi Carti`. For the GOAT artist Future, +I want to create a folder for each of his albums. For example, for an entry like +`SUPER SLIMEY, Future` I would expect a directory called `SUPER SLIMEY` to be created. + +**TLDR**: You need to fetch the list from the web, filter out the albums we want, +trim out the album name, and then make a directory for each one, all in one line. + +```bash +$ cat albums.txt +... +Drip or Drown 2, Gunna +Playboi Carti, Playboi Carti +DS2 (Deluxe), Future <-- GOAT album detected! +Drip Harder, Lil Baby +The WIZRD, Future <-- GOAT album detected! +What a Time To Be Alive, Drake +... + +# After our magic one liner... + +$ ls +'DS2 (Deluxe)' 'The WIZRD' ... +# We got our new directories! +``` + +**Hints:** +- What common text manipulation commands can help you solve this? +- As always, be aware that there isn't one unique solution to this problem! + +Submit your one line solution on Gradescope! + # Question 4 (Extra) This question is optional but it's quite fun and you should do it if you have the time! @@ -200,6 +278,8 @@ Submit your solutions on Gradescope! There'll be some extra feedback questions a You may want to look into `dd`[^dd] and the `iflag=fullblock` argument, `seq`, and `/dev/random`[^rand]. + + [^dd]: `dd` is a command used to copy files.[^dd2] It's most commonly used to clone data from one device to another, such as when you want to [generate a bootable Linux USB drive][usb]. [^dd2]: "But wait," a nearby straw-man asks, "isn't that what `cp` does?"[^dd3] @@ -221,3 +301,10 @@ You may want to look into `dd`[^dd] and the `iflag=fullblock` argument, not exist in certain machines and gathering entropy may be prohibitively long. Thus, `/dev/urandom`, or "unlimited random", is a useful source when such randomness is not cryptographically critical. + +[1]: https://en.wikipedia.org/wiki/Shebang_(Unix)#Magic_number +[2]: http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html +[3]: https://linux.die.net/man/1/cut +[4]: https://linux.die.net/man/1/grep +[5]: https://linux.die.net/man/1/xargs +[6]: https://en.wikipedia.org/wiki/Man_page \ No newline at end of file From 5b3ca08f84a220d022395630548dd9706cff8793 Mon Sep 17 00:00:00 2001 From: Lance Math Date: Sat, 22 Jan 2022 06:18:18 -0800 Subject: [PATCH 2/2] added a1 skeleton --- labs/a1.md | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/labs/a1.md b/labs/a1.md index af3fb2d3..5537b3cf 100644 --- a/labs/a1.md +++ b/labs/a1.md @@ -92,6 +92,10 @@ The `LIST` can either be something like a directory with multiple files or a fil ## Useful Commands Some commands which could be useful in completing the lab. Of course, there are many approaches to the problems, and using these commands is not required. +### `cat` + +`cat` [prints][8] files to the [standard output][7]. Very useful for printing something to pipe into other commands! + ### `cut` `cut [options] [filename]` extracts certain parts of a file (or piped-in input) based on the [arguments][3] that are used. A few which might be useful: @@ -106,6 +110,16 @@ Some commands which could be useful in completing the lab. Of course, there are `grep [pattern] [filename]` [filters out][4] and returns lines from a file (or piped-in input) that contain the specified pattern. +### `sed` + +`sed` can do [many things][9], such as editing strings and matching regex. We can use `sed` to replace one pattern with another pattern as follows: + +``` sed 's///g ' ``` + +sed` can also take piped-in input from something else instead of an explicit input. + +The `g` at the end tells `sed` to replace all ocurrences of the pattern; it can be omitted if we want to replace only the first ocurrence of a pattern, or replaced with a number to replace only a certain number of occurrences. + ### `xargs` `xargs` lets us apply a command to an output redirected from a pipe. For instance, `output | xargs command` would apply `command` to `output`. Some useful [options][5]: @@ -195,6 +209,8 @@ $ ls -lAh tmp | grep .txt | wc -l # Gets the number of lines in ls which contain 26 ``` +**Hint:** Need help looping through files? See this week's participation assignment on [Gradescope][10] for more hints on `for` loops! + for bonus points, instead of using something like `sed` to affect the rename, use [shell parameter expansion][1]. @@ -238,7 +254,9 @@ $ ls ``` **Hints:** +- If it's easier, download the above file and work on it in your VM so you don't have to process output from `wget`. - What common text manipulation commands can help you solve this? +- Consider the commands `cat | grep ___ | cut ___ | sed ___ | xargs ___` (though you don't *have to* use them...) - As always, be aware that there isn't one unique solution to this problem! Submit your one line solution on Gradescope! @@ -307,4 +325,8 @@ You may want to look into `dd`[^dd] and the `iflag=fullblock` argument, [3]: https://linux.die.net/man/1/cut [4]: https://linux.die.net/man/1/grep [5]: https://linux.die.net/man/1/xargs -[6]: https://en.wikipedia.org/wiki/Man_page \ No newline at end of file +[6]: https://en.wikipedia.org/wiki/Man_page +[7]: https://linuxhint.com/bash_stdin_stderr_stdout/ +[8]: https://linux.die.net/man/1/cat +[9]: https://linux.die.net/man/1/sed +[10]: https://www.gradescope.com/ \ No newline at end of file