17 posts tagged with "bash"

In my previous article; Linux - Take advantage of the number of CPUs you have; start concurrent jobs, we've seen how to start jobs in parallel.

The next cool thing is to show a progression bar in your console. This has a number of advantages, including a clear view of what's been done and what's still to be done, as well as an attractive interface.

Months ago, I've found this french blog post in my RSS feeds: https://xieme-art.org/post/bash-avance-barre-de-progression/ and, just, wow!!!

Let's play with it.

In my professional activity, I've been faced with the following requirement: process each line of a CSV file and make a POST API call to upload a document.

One line of the CSV contained information that needed to be communicated to an API service, and each line corresponded to a PDF file. So if there are 1000 lines in the CSV file, I have to make 1000 API calls to upload 1000 PDFs.

I wrote my script in Linux Bash and then it was time to optimise: not just one API call at a time, but as many as possible.

Let's how we can start more than one task at a time using Linux Bash.

When writing Bash scripts, I'm always putting some description block in front of any functions like I do in any language (think to PHP Docblock).

With PHP, there are a few tools like phpDocumentor for extracting these blocks and generate documentation but do such tools exist for Bash? I don't know, I haven't found any.

So I've written a small Bash script to accomplish this i.e. parse any .sh file present in a folder, extract doc blocks and create one markdown document for any retrieved script. Each function's documentation will then be copied in Markdown, then a table of contents will be appended and, finally, a generic readme.md file will display the list of markdown files retrieved.

Still faced with the problem of having to compare two versions of the same script (see the article Linux - Comparing two folders/files in the console), this time we're going to consider that the file we need to compare is a Bash script and that we have two versions of it.

And that we may have made the two files evolve in different ways, i.e. that one or other, or even both, of the files may have been modified.

The aim is therefore to compare the versions and highlight the differences.

In a previous article, we've seen a simple CLI command to display the list of functions present in a script but what about a Bash script that will scan an entire folder; retrieve any .sh scripts and check if functions are sorted in the files?

Natively, Linux has a command-line tool called diff for comparing two folders or files. Comparing two folders is quite simple: diff folder_1 folder2. And it's no more complicated for two files: diff file_1 file2.

However, when you want to do this in a slightly more industrialized way (launch a very large number of comparisons to compare two versions of the same project, for example), the use of a few flags and snippets comes in handy.

In my previous article Bash - Script to add logging features to your script, I've shared a way to write information in a logfile.

By running ls -alh /tmp you will get the list of all files in the /tmp folder and display the list on your console. By running ls -alh /tmp >> application.log you won't see the list in your console since everything will be written in the application.log file.

How can we display the output of a command like ls f.i. both on the console and in a logfile?

Also read Bash - Echo on the console and in a logfile in the same time

When you write Bash scripts and certainly when you foresee running them in a cron, you should implement a logfile. Every action fired by your script should be logging somewhere so you can start the script in a non-interactive mode and in case of need, consult the last logfile.

Below is a script I've developed in the form of a library, which means you can easily include it in your existing code without having to change anything.

You just need to include the file in your script (i.e. add a source log.sh line) then here and there foresee a log::write "Something to log". Easy no?

As you probably know, CTRL+R in the console will give you access to your HISTORY i.e. you'll retrieve the list of the commands you previously typed in your console. Just like using UP or DOWN keys but with a very small search engine.

And it makes the job but, honestly, this is really basic, no?

The Fuzzy Finder (aka fzf) command line utility will explode the possibilities linked to searching in the history but, in reality, this is just one of the consequences of installing fzf, which is so much more powerful.

Let's have a look.

This is a very common source of problems using .env files: you've two or more different .env file like .env and .env.example.

You're a programmer and coding a new amazing feature. You're adding one or more new environment variables to your local .env file and everything is working fine on your computer.

Why? Because the variable(s) you've added have been added in your local .env file, on your computer only.

As you know, you have to create the variables in the .env.example file too but let's be honest, nobody thinks about it.