Creating a CSV-Report with Apache and Jenkins

Intro

Creating a CSV out of a list of data (test results or any other kind) in Java, isn’t a big thing these days. But creating a file in Jenkins, feeding some comma separated values to it and then saving that file to your Jenkins build, is a bit trickier if you have never done it before. So here is the challenge I recently got:

Out of some test result data, create a CSV file which can be opened in MS Excel and make that file available in the specific Jenkins build job so that it can be downloaded as a test-result artifact by the people of department X.

Cool challenge isn’t it! I love doing such stuff! Let’s cut this in some eatable pieces by using the pomodoro technique:

• Structure the Data
• Create a CSV file in Jenkins and make it available as artifact
• Write the data to the CSV file
• Download the CSV file from the Jenkin3s build

Sounds easy, right? Well, it does, but if you haven’t done that before, it may take you a while to figure out how to put the puzzle pieces correctly together in code. Let’s have a look at the solution I (and a working mate, 4thank you buddy!) came up with.

Structure the Data

Data in its raw form is like a pile of unsorted stones. It is hard to process raw data. First we need to structure it to become handy. Putting data into a data structure, is a way of organising data so that it can be accessed and used efficiently.

(Source)

For our example we are going to use a Java ArrayList which implements the List interface and add some test-result data to it.

Great, that was easy right? Now that we have our little structured data set in place, let’s have a look at how we can create a CSV file in Jenkins.

Create a CSV file in Jenkins and make it available as artefact

(Source)

These days, a Jenkins pipeline is implemented within a Jenkinsfile. The Jenkinsfile is version controlled alongside all the rest of the code. A Jenkins-Job will then read the Jenkinsfile from the repo, build up the pipeline accordingly and execute the specified stages. To create a file in a Jenkinsfile we can do the following: (For reusability purposes, we first create a variable for the file name.)

Then, as part of a stage, we write the file and save it in the current build workspace as artifact:

Writing the Data to the CSV file

To write data to a CSV file in Java, we can go two ways. We can either use the java.io package and write the code on our own or we can use a third-party library. Let’s have a look at how we could implement this with our own bare hands.

Do It yourself

As we already have our data at hand, we need some sort of magic to convert this List of Strings into a line of comma separated values. This can be done with the help of the Streaming API:

Before we go on and write that CSV String into a file, we also have to think about escaping of special characters which could appear in hour data (e.g. single or double quotes, new line, backslash, tab, backspace) and which character encoding we want to use. Let’s keep things simple and focus only on escaping commas (,), quotes ('or") and new lines (n) in our data String. Fields containing commas or quotes will be surrounded by double quotes and double quotes will be escaped with double quotes. We'll eliminate new lines and replace them each with whitespace. For this we create the following method:

With all the things from above in hand, we can now finally write the data to the CSV file:

Writing the implementation for creating a CSV file can quickly become quite complicated when we start thinking of how we have to handle special characters and character encoding. Luckily there are a few third-party libraries available for working with CSV files in Java which handle this for us flawlessly. Let’s have a look which CSV library I have chosen and how you can write a CSV with it.

Using Apache Commons CSV

(Source)

The Apache Commons CSV library has many useful features for creating CSV files. The CSV component of the Apache commons project is under the umbrella of the Apache Software Foundation and is actively maintained; that is why I have chosen it.

Also here, as we already have the data ready, we just need to do the implementation for writing a CSV file:

As you notice, we don’t have to bother with any additional implementation for dealing with the escaping of special characters and character encoding. We just have to specify which encoding we want to use and that’s it! The rest is handled by the library itself.

Download the CSV file from the Jenkins build

Finally we can run our Jenkins-Job and while waiting, we drink a cup of tea. Once the job has finished, we head over to the Artifacts section, download the .csv file and open it in MS Excel. Job done!

Of course this is probably not the best solution on earth, but we have had a lot of fun to come up with this implementation and make our internal customers happy with it!