QA Staff Augmentation and Software Testing Provider

What we do

Users are the first to abandon applications with usability or functional defects. Mastering

Staff augmentation is a strategy for diminishing costs and filling skills gaps in

Quality assurance (QA) refers to a systematic process that aims to ensure the

Who are we

Pragmatic – testing of software applications

Pragmatic is a company founded in 2013 in Sofia, Bulgaria.
The main focus of the company is to provide quality service – testing of software applications.
In addition, we offer a wide range of courses related to the QA / Tester profession.

Read more

We are at your disposal

Request pilot project

Read more

A Team of
Distinguished Pros

Over the years of our business’ existence, we’ve been solving thousands of puzzling business management issues… Still, each time we found a working solution, and not a workaround! This is true when we recall, that over 3000 companies has been our satisfied clients since 2001…

Our team

Aleksandar Karamfilov

Aleksandar Karamfilov

12 years of experience in IT industry 11 years as a QA/Test engineer Bachelor degree in Computer science Full ISTQB
Milen Strahinski

Milen Strahinski

13 years of experience Bachelor degree in computer science

Let the Number Speak
for Us and Our Clients

In our portfolio
we offer


We educated
more than


Working in
the industry

of them

In our company work


Recent Blog Entries

Model-Based Testing with GraphWalker

Model-Based Testing with GraphWalker

Model-Based Testing with Graph Walker Functional testing takes huge amount of our time as testers. We try to…

Reducing Environment Combinations with Microsoft PICT

Reducing Environment Combinations with Microsoft PICT

What is N-Wise Testing? N-wise testing has the aim of testing all the possibilities of any random combination…

JUnit 5 (Jupiter) is here!

JUnit 5 (Jupiter) is here!

Short catchup on how lambda expressions became reality in Java

Since JUnit 5 is already here let’s have a look on it and enjoy its new approach and functionalities. In order to be able to consume everything properly we will need to have a short reminder of how lambdas work in Java. A simple classical implementation of Java is for example the FileFilter interface that looks as follows:

public interface FileFilter {
     boolean accept(File pathname);

and one of the classical approaches for implementation before Java 8 was the following one:

public class JavaFilesFilter implements FileFilter {
     public boolean accept(File pathname) {
          return pathname.getName().endsWith(“.java”);

Then when it comes to using it the result code was as follows:

JavaFilesFilter filesFilter = new JavaFilesFilter();
File directory = new File(“e:/tmp”);
File[] javaFiles = directory.listFiles(filesFilter);

which pretty much as a result return to us an array which contains all the files that end with *.java in their filename and are inside directory e:tmp.

After that because we do not wanted to have plenty of classes that implement interfaces and they stay in our project structure forever, we started using the anonymous class implementation which helped us avoiding the creation of a class whenever we needed some FileFiler for example. So the result was the following one(and that was the most used way till Java 7):

FileFilter filesFilter = new FileFilter() {
   public boolean accept(File pathname) {
        return pathname.getName().endsWith(“.java”);

File dir = new File(“e:/tmp”);
File[] javaFiles = dir.listFiles(filesFilter);

So the concept of functional interfaces appeared which is pretty much an interface with only one abstract method. They are divided in 4 main categories:

public interface Function<T, R> {
     R apply(T t);

public interface Supplier { 
     T get();

public interface Consumer {
     void accept(T t);

public interface Predicate {
     boolean test(T t);

So since Java 8 this is how the above implementation with the anonymous class will look like by using lambda expression:

FileFilter filesFilter = (File pathname) -> pathname.getName().endsWith(“.java”);

as you can see with a single line you can have the filesFilter that you can use afterwards.

 Now lets get into JUnit 5 

1) How the JUnit Platform is organized

So the JUnit 5 platform mainly consists of the following jars:

junit-platform-console – for discovering and running tests from the console
junit-platform-engine – an API for test engines
junit-platform-launcher – an API for launching and configuring tests typically used by IDEs and build tools like Maven and Gradle
junit-platform-runner – for executing tests in a Junit 4 environment
junit-platform-gradle-plugin – for discovering and executing tests using Gradle
junit-platform-surefire-provider – for discovering and executing tests using Maven Surefire plugin
junit-jupiter-api – an API for writing tests and extensions (needed compile time)
junit-jupiter-engine – the engine implementation to run the tests(i.e. at runtime)
junit-jupiter-params – for parametrized tests
junit-jupiter-migrationsupport – provides support to some JUnit 4 rules to the new extension model of JUnit(5) Jupiter
junit-vintage-engine – the engine implementation to execute tests written in JUnit 3 or JUnit 4(and of course you will need the Junit3/4 jars as result too)

and if you wonder why exactly JUnit Jupiter, this is because the 5th planet in our solar system is exactly Jupiter. 🙂

2) What do we need to start using it

In order to use it with Maven you will need the following dependencies added to your pom.xml:

junit-jupiter-api – for compile time tests writing
junit-jupiter-engine – for running the tests, i.e. runtime
junit-jupiter-params – for the parameterized tests

and also the following build plugins again in your pom.xml:

maven-compiler-plugin – make sure to override it from 1.5 to 1.8 version of the compiler-plugin
maven-surefire-plugin – this is the plugin that will run our tests, but we will add some configurations to it later

Have in mind that by default the maven-surefire-plugin will only pick to run test classes that match one of these names:


of course you configure that alternatively if needed as this is the default configuration.

3) Most used annotations

In case you would like to have a look on all the available annotation in JUnit 5 you need to just open their official documentation and have a look under:

But basically the annotations are as follows:

@Test – indicates that a method is a test

and the lifecycle annotations:

@BeforeEach – this will run before each @Test
@AfterEach – this will run after each @Test
@BeforeAll – this will run before ALL @Test are executed
@AfterAll – this will run after ALL @Test are executed

So to put it in an example you can simply create methods like:

public void nameOfTheMethod(){

and that will be a test case method, and in the same manner you can create more methods in the same class and put @BeforeEach or @AfterEach or some of the others and they will behave exactly as described above.

For a demonstration take a look under package bg.pragmatic.salarycalc.lifecycle.annotation.and.executions and bg.pragmatic.salarycalc.simpletests of GitHub repo: https://github.com/mstrahinski/junit5examples.git

4) Tagging

While creating tests you would most probably like to tag them in some way, so at some point in order to execute them you normally don’t want to execute all of them, but only a specific group of tests(or in other words some tests suite). So together with the @Test annotation you can put the @Tag annotation in the following manner to tag your test, example:

public void nameOfTheMethod(){

and when it comes to a run with the maven-surefire-plugin, you will be able to simply add the following configuration in your pom.xml:

      <includeTags>regression, sanity</includeTags>

that way it will run only the tests that are tagged with “regression” and “sanity” and wont run any tests that are tagged with “slow”. However that way they will be hardcoded and you will need to edit the pom.xml on each run, so another alternative to provide the tagged tests that you want to be run is to give it to the command itself and NOT hardcode it in the pom.xml, i.e.:

g.pragmatic.salarycalc.tags of GitHub repo: https://github.com/mstrahinski/junit5examples.git

5) Hierarchies

In JUnit 5 they have introduced the ability to create test hierarchies

@Nested – Denotes that the annotated class is a nested, non-static test class. @BeforeAll and @AfterAll methods cannot be used directly in a @Nested test class unless the “per-class” test lifecycle is used for the nested class. Such annotations are not inherited.
@DisplayName – Declares a custom display name for the test class or test method or in other words normally you see the name of the test method as a name of the test in the results, but if you put a @DisplayName(“some display name”) in the annotations of the test it will have a more descriptive and readable name. Such annotations are not inherited.

For a demonstration take a look under package bg.pragmatic.salarycalc.hierarchies of GitHub repo: https://github.com/mstrahinski/junit5examples.git

6) Enabling and Disabling tests

@Disabled(“Some reason for disabling”)
works on Method level i.e. one test for example
work also on Class level i.e. disable all the tests in some class

@DisabledOnOs( {WINDOWS, MAC} ) – the test will not run if the OS is from the mentioned ones
@EnabledOnOs({WINDOWS, MAC}) – important to understand is that it means it’s enabled on WINDOWS and MAC and DISABLED for everything but them.

For a demonstration take a look under package bg.pragmatic.salarycalc.disabled of GitHub repo: https://github.com/mstrahinski/junit5examples.git

7) Assumptions

Sometimes you want to stop the execution of a test at some point based on a condition, you can even put it in the @BeforeEach method

Assumptions are not like assertions and the result of it are NOT test failure – they only abort the test


For a demonstration take a look under package bg.pragmatic.salarycalc.assumptions of GitHub repo: https://github.com/mstrahinski/junit5examples.git

8) Repeating a Test

To repeat a test multiple times you simply put the @RepeatedTest annotation above it and the number of repetitions are fixed and cannot be changed at runtime
Each repetition of a test is the same as normal test and it has a full lifecycle support
You can also give a different custom name of each repetition, supported placeholders are: {displayName}, {currentRepetition}, {totalRepetitions}

Predefine formats:
{displayName} :: repetition {currentRepetition} of {totalRepetitions}

RepeatedTest.SHORT_DISPLAY_NAME //this is the default format if nothing is specified
repetition {currentRepetition} of {totalRepetitions}

We can have also RepetitionInfo object available during our test execution in @RepeatedTest, @BeforeEach, @AfterEach

@RepeatedTest(value = 10, name = “{displayName} – {currentRepetition} of {totalRepetitions}”)
public void someTestMethod(RepetitionInfo repetitionInfo) {

If you add the @Test annotation, that will be one additional run

For a demonstration take a look under package bg.pragmatic.salarycalc.repeatedtest of GitHub repo: https://github.com/mstrahinski/junit5examples.git

9) Parametrized Tests

With that approach you can run tests multiple times with different arguments, or in other words also known as data driven tests. The annotation @ParameterizedTest is used.

Have a lifecycle like regular test methods, but require at least one data source and the dependency junit-jupiter-params in our pom.xml.

It’s also part of the Experimental APIs of JUnit 5, so changes might happen in the future. You can set custom display names using the following placeholders:
{index} – the current invocation index based on the data source, starts from 1
{arguments} – the complete comma separated arguments list
{0}, {1}, {2}, … – use each individual argument, starts from 0

You can inject two type of parameters:
TestInfo – with information about the current test/container
TestReporter – can only be injected into @BeforeEach, @AfterEach, @Test

Be aware that you can inject TestInfo and TestReported only AFTER the parameters that are injected by the @ParameterizedTest annotation

The predefined sources can be:
@ValueSource – for test methods with single parameter(ints, doubles, Strings …)

@EnumSource – run a test with the values of an enum, optional params – names and mode

@MethodSource – refer to one or more methods, must be static unless using PER_CLASS lifecycle
Tests with single parameter methods – return Stream of parameter or primitive type
Tests with multiple parameters – return Stream, Iterable, Iterator or array of Arguments

@CsvSource – delimiter as a parameter, and single quote ‘ used as quote character

@CsvFileSource –takes one or more csv files, you can define encoding, line separator and delimiter, double quote “ used as quote character

@ArgumentsSource – that way you can use your own sources, you only need to implement the ArgumentsProvider interface

For a demonstration take a look under package bg.pragmatic.salarycalc.parameterizedtest of GitHub repo: https://github.com/mstrahinski/junit5examples.git

10) Dynamic Tests

A more powerful alternative of @RepeatedTest

So normally we create tests with @Test and define the behavior of each test at compile time and we cannot change its behavior at runtime, this is where the dynamic tests come. To create a dynamic test you need a method annotated with @TestFactory, which is not a test method by itself but a factory of multiple tests (must NOT be private or static as they wont get executed). It’s marked as Experimental API so have in mind that it might change in the future. JUnit 5 prefers extensions over features principle. As a result, the main aim of dynamic tests is to provide an extension point for third party frameworks or extensions.

To create dynamic tests, the @TestFactory uses as data source Collection, Iterable, Iterator or a Stream. All of the mentioned MUST contain an element of type DynamicNode(which is the parent of DynamicContainer and DynamicTest)

DynamicContainer – Has a display name and contains rather Iterable (or Stream) of DynamicNode, in other words can contain other DynamicContainer or DynamicTest

DynamicTest represents a test generated at runtime – Has a display name and Executable (which is a functional interface that wraps the code of the test, in other words we can use lambdas to write the test implementations)

Be aware that dynamic tests do not regard the test lifecycle and @BeforeEach and @AfterEach are NOT executed for each dynamic test, but only once

For a demonstration take a look under package bg.pragmatic.salarycalc.dynamictest of GitHub repo: https://github.com/mstrahinski/junit5examples.git

QA of the Year Award (2019)

QA of the Year Award (2019)

Our CTO Aleksandar Karamfilov was awarded with QA of the Year Award for 2019. QA of the Year…

a call back

Whatever types of questions you’ll have, feel free to get in touch with us!