how people do unit testing

Thread Starter

bug13

Joined Feb 13, 2012
2,002
Hi team

How do people do unit testing? Are there any tools available? I just normally write a piece of code in a separate IDE and run some test myself. Keen to know how other people do it.

Thanks team!
 

Thread Starter

bug13

Joined Feb 13, 2012
2,002
Since this is in the Programming forum, I assume you're talking about testing software.

If you're the author or have access to the code, the typical method is to do white box testing. If you don't have access to the code, you do black box testing.

http://softwaretestingfundamentals.com/differences-between-black-box-testing-and-white-box-testing/
Yes I am the author and have access to the code. How do you do white box testing? Do you use any tools/techniques/strategy etc...?
 

402DF855

Joined Feb 9, 2013
271
Generally a unit test would target a specific module, a .c file or perhaps a .h/.cpp pair. A "test harness" is written which is used in a standalone project to invoke the features of the module repeatedly and ensuring the correct behavior is observed. Failures of the code should somehow be indicated at the end of the test.

So, for example, the Queue discussed in the other thread would be tested by creating queues, inserting and removing, and measuring the results as the harness executes.
C:
Queue_ptr ptr = CreateQueue();

check_condition(ptr->num_elements == 0);
// More tests follow
In its simplest form the check_condition function would keep track of the number of failures. Even for embedded software a great technique is to write and debug the test harness on a PC; then build and run it on the target hardware or a simulator thereof.
 

Thread Starter

bug13

Joined Feb 13, 2012
2,002
So, for example, the Queue discussed in the other thread would be tested by creating queues, inserting and removing, and measuring the results as the harness executes.
C:
Queue_ptr ptr = CreateQueue();

check_condition(ptr->num_elements == 0);
// More tests follow
Clean and simple explanation, thanks a lot for the reply!! :)
 

MrSoftware

Joined Oct 29, 2013
2,188
It depends on what your code does as to how you would test it. For example I worked on a product that produced physical output files. We wrote a bunch of tests that with a specific input should have given a specific output. Write the tests to cover all of the boundary conditions, and some tests that cause errors too. We ran the tests to produce known good output files and output logs and saved them. Then every night a script would check out and build the latest code, run the tests, diff the results and the logs against what was expected and email the result to the developers. Our code was multi-platform so the same tests would run on Windows, Linux, Irix, AIX and Solaris. The source control had a java command line interface that was the same for all platforms, and I wrote all the scripts in perl, so the same code ran on all platforms with some minor platform specific sections, such as how to call the compilers since the compilers were all different.

It's good if the person who developed the code is NOT the person writing all of the tests. Write some of the tests for areas that you think are susceptible to errors over time, but have someone else write some too.
 

Thread Starter

bug13

Joined Feb 13, 2012
2,002
It's good if the person who developed the code is NOT the person writing all of the tests. Write some of the tests for areas that you think are susceptible to errors over time, but have someone else write some too.
Thanks for the reply, I think that's a good idea!
 

MrSoftware

Joined Oct 29, 2013
2,188
Another thing I like to use are asserts. An assert is only run in a debug build and is not compiled into a release build, and if the test statement fails the code will crash (or stop the debugger) right there so you can instantly see what failed. These are especially useful when errors won't be obvious. This isn't the best example, but you can see how not catching this error could lead to unexpected results that might not be easy to track down in a big project:

C++:
uint32_t iBigNumber = GetSomeResultExpectedToBeSmallerThan255FunctionReturns32bitResult();
ASSERT(iBigNumber <= 255);  // Debugger will stop here if iBigNumber > 255
uint8_t iSmallNumberToPassTo8BitProcessor = (uint8_t)iBigNumber;   // This would run but might cause unexpected output that is difficult to track down if number is > 255
You can also get a code coverage analysis tool to help you verify that your tests exercise every line of code. It won't necessarily help you test every possible scenario, but at least you can hit every line.
 

402DF855

Joined Feb 9, 2013
271
Similarly when undertaking a very thorough unit testing activity I'll shoot for "logic coverage", i.e. making sure all of the branches in the emitted assembly language are fully covered.
Code:
002086E0 E92D41F3 * push        { r0, r1, r4, r5, r6, r7, r8, lr }
002086E4 E1A04002 * mov r4, r2
002086E8 E1A06000 * mov r6, r0
002086EC E1A08001 * mov r8, r1
002086F0 EBFFF9DD * bl  206e6c ; 
002086F4 E3540000 * cmp r4, #0
002086F8 C3500000 < cmpgt       r0, #0
002086FC D3A05001 > movle       r5, #1, 0 ; 0x00000001
00208700 C3A05000 < movgt       r5, #0, 0 ; 0x00000000
00208704 D3A04000 > movle       r4, #0, 0 ; 0x00000000
00208708 DA00001B > ble 20877c 
0020870C E1A07000 * mov r7, r0
00208710 E28D0004 * add r0, sp, #4
00208714 EBFFFD3D * bl  207c10 
00208718 E1550008 * cmp r5, r8
0020871C AA000005 x bge 208738 
00208720 E5963000 * ldr r3, [r6]
00208724 E28D0004 * add r0, sp, #4
00208728 E7D31005 * ldrb        r1, [r3, r5]
0020872C EBFFFD67 * bl  207cd0 
00208730 E2855001 * add r5, r5, #1 ; 0x1
00208734 EAFFFFF7 * b   208718 
00208738 E0844008 * add r4, r4, r8
0020873C E1540007 * cmp r4, r7
00208740 AA000005 x bge 20875c
This listing was generated after some test cases have finished. The asterisk indicates an instruction has been run. < and > indicate a conditional has been executed but only one of two conditions were seen. The x indicates both conditions (true and false) were covered.

Based on what you see there are conditional branches that have not been covered by the test cases and therefore my work is not finished.
 

Thread Starter

bug13

Joined Feb 13, 2012
2,002
Similarly when undertaking a very thorough unit testing activity I'll shoot for "logic coverage", i.e. making sure all of the branches in the emitted assembly language are fully covered.
Code:
002086E0 E92D41F3 * push        { r0, r1, r4, r5, r6, r7, r8, lr }
002086E4 E1A04002 * mov r4, r2
002086E8 E1A06000 * mov r6, r0
002086EC E1A08001 * mov r8, r1
002086F0 EBFFF9DD * bl  206e6c ;
002086F4 E3540000 * cmp r4, #0
002086F8 C3500000 < cmpgt       r0, #0
002086FC D3A05001 > movle       r5, #1, 0 ; 0x00000001
00208700 C3A05000 < movgt       r5, #0, 0 ; 0x00000000
00208704 D3A04000 > movle       r4, #0, 0 ; 0x00000000
00208708 DA00001B > ble 20877c
0020870C E1A07000 * mov r7, r0
00208710 E28D0004 * add r0, sp, #4
00208714 EBFFFD3D * bl  207c10
00208718 E1550008 * cmp r5, r8
0020871C AA000005 x bge 208738
00208720 E5963000 * ldr r3, [r6]
00208724 E28D0004 * add r0, sp, #4
00208728 E7D31005 * ldrb        r1, [r3, r5]
0020872C EBFFFD67 * bl  207cd0
00208730 E2855001 * add r5, r5, #1 ; 0x1
00208734 EAFFFFF7 * b   208718
00208738 E0844008 * add r4, r4, r8
0020873C E1540007 * cmp r4, r7
00208740 AA000005 x bge 20875c
This listing was generated after some test cases have finished. The asterisk indicates an instruction has been run. < and > indicate a conditional has been executed but only one of two conditions were seen. The x indicates both conditions (true and false) were covered.

Based on what you see there are conditional branches that have not been covered by the test cases and therefore my work is not finished.
My assembly is very limited, but I am interested on what tool you use to get those information?
 

402DF855

Joined Feb 9, 2013
271
That specific one is an ARM simulator I wrote for my own purposes. No, you can't have it. :)

The coverage features I added were based on my experience years ago with a Z8000 simulator we used developing commercial avionics. I don't recall where it came from; it may have been developed in-house, or purchased.
 

dl324

Joined Mar 30, 2015
16,839
The type of testing needed also depends on the code.

I wrote some code for Arduino where I was using multiple timer interrupts and I made sure I disabled interrupts when variables longer than a byte were being updated.
 

Thread Starter

bug13

Joined Feb 13, 2012
2,002
The type of testing needed also depends on the code.

I wrote some code for Arduino where I was using multiple timer interrupts and I made sure I disabled interrupts when variables longer than a byte were being updated.
I thought you only need to disable the interrupts when updating the none atomic variables use inside and outside the interrupt? The none atomic variables are OK if they are used outside the interrupt. (Assuming necessary registers are saved and restored on entry and exit interrupt)
 

dl324

Joined Mar 30, 2015
16,839
I thought you only need to disable the interrupts when updating the none atomic variables use inside and outside the interrupt?
Interrupts are disabled by default within an interrupt service routine, so you wouldn't normally need to disable interrupts when updating non-atomic variables in an ISR. But you need to do it everywhere else. If you forget to do that, you'll be faced with some very difficult to debug problems.
 
Last edited:

Thread Starter

bug13

Joined Feb 13, 2012
2,002
Interrupts are disabled by default within an interrupt service routine, so you wouldn't normally need to disable interrupts when updating non-atomic variables in an ISR. But you need to do it everywhere else. If you forget to do that, you'll be faced with some very difficult to debug problems.
That is what I am trying to say:
C:
/* Assuming a 8-bit architect like AVR */
volatile uint16_t var_isr;  /* use inside and outside ISR */
uint16_t var_normal;        /* use ouside ISR only        */

Interrupt someISR(){
    SaveAffectedRegisters();
    var_isr++;    /* update variable */
    RestoreAffectedRegisters();
}

void main(){
    
    for(;;){
        /*!    it need to has atomic access as it's
         *     updated inside an ISR
         */
        ISR_DIS();
        var_isr = var_isr * 2;
        ISR_EN();
        
        /*!    This does not has atomic access as it's
         *     not used inside any interrupt. It's accessed
         *     outside ISR only
         */
        var_normal = var_normal * 2;
    }
}
 

Thread Starter

bug13

Joined Feb 13, 2012
2,002
That specific one is an ARM simulator I wrote for my own purposes. No, you can't have it. :)

The coverage features I added were based on my experience years ago with a Z8000 simulator we used developing commercial avionics. I don't recall where it came from; it may have been developed in-house, or purchased.
That's a hell of skills!
Not intend to try to do what you have done. But how (in top level) do you write an simulator in general? Assuming the host is an PC?
 

402DF855

Joined Feb 9, 2013
271
The first step is to write a disassembler. So be able to load a HEX or ELF or other executable into memory and then decode the instructions. Once that is working, you create implementations for all of the instructions the disassembler recognizes, operating on the memory spaces and registers.

I've written a simulator for at least 5 processors. It's a good way to learn the assembly language. And then I have functional simulator for development purposes (e.g. unit test). Of course simulators are available for most processors, but I'm not familiar with them much since I write my own.
 

Thread Starter

bug13

Joined Feb 13, 2012
2,002
you create implementations for all of the instructions the disassembler recognizes, operating on the memory spaces and registers.
So you implement all the registers as of a real physical device and memory spaces, that is pretty cool! Do you have something to simulate the IOs as well?
 
Top