Memory leak when used in C++

[SigmaOrionis]

Describe the bug
Memory leak

To Reproduce
Multiple edit distance check.

Expected behavior
Keep memory at the same level

Environment (please complete the following information):

• OS: Windows 10
• Version c++ latest

Additional context
Using edit distance as described

[Martinsos]

Hi @SigmaOrionis, thank you for reporting this!

I will however need some more details in order to be able to try to reproduce this.
Do you have a minimal code example that I can try? If not, at least a code snippet from your codebase showing the usage of Edlib?
What do you mean by "multiple edit distance check" -> did you run it in parallel? Or just check after check?
How do you know there is a memory leak, how did you detect it / how does it manifest?

[SigmaOrionis]

Hi, multiple thread call to:

auto sub = translatedSlim.substr(0, Input.length());
auto ed = edlibAlign( Input.c_str(), Input.length(), sub.c_str( ), sub.length(), edlibDefaultAlignConfig( ) ).editDistance;

detected in release build attached to console application statically, or as attached dll accessed through CLR.

Pozz ;)

[Martinsos]

I never used edlib in multiple thread call situation, but I would expect it to behave nicely / not differently than when normally called.
I don't have much experience with multi threaded execution in cpp, could you give me some suggestions how to replicate this?

One thing to keep in mind: you should call freeEdlibAlignResult() on result of edlibAlign when you don't need it any more. Although I think in your specific case that might not be the problem (because you are using default align config), but you should still do it to be safe.

Also, how did you detect that memory leak is happening, and that it is coming from edlib, could you provide more details on that? How much is memory allocation growing, where is it coming from, anything?

[SigmaOrionis]

Bok Martin,

call to edlibAlign is scoped and as such by all coding standards should free itself. It does not matter that I am calling it from multiple threads, it would leak from single thread. Leak is detected by monitoring memory allocation in my case by using Visual Studio 2019 memory allocation monitor. You know what, here you go:

// Testing.cpp : This file contains the 'main' function. Program execution begins and ends there.
//

// debug_new.cpp
// compile by using: cl /EHsc /W4 /D_DEBUG /MDd debug_new.cpp
#define _CRTDBG_MAP_ALLOC
#include <cstdlib>
#include <crtdbg.h>

#ifdef _DEBUG
#define DBG_NEW new ( _NORMAL_BLOCK , __FILE__ , __LINE__ )
// Replace _NORMAL_BLOCK with _CLIENT_BLOCK if you want the
// allocations to be of _CLIENT_BLOCK type
#else
#define DBG_NEW new
#endif

#include <stdio.h>      
#include <stdlib.h>     
#include <time.h>       
#include <string>
#include "edlib.h"

std::string getRandomString( )
{
    int randlen = rand( ) % 10 + 1; // rand between 1 and 10
    char randChar;
    std::string out;

    for( int i = 0; i < randlen; i++ )
    {
        randChar = rand( ) % 122 + 48;
        out.push_back( randChar );
    }

    return out;
}

int main()
{
    srand( time( nullptr ) );

    std::string one;
    std::string two;

    for( int i = 0; i < 1000; i++ )
    {
        one = getRandomString( );
        two = getRandomString( );

        auto ed = edlibAlign( one.c_str( ), one.length( ), two.c_str( ), two.length( ), edlibDefaultAlignConfig( ) ).editDistance;
    }

    _CrtDumpMemoryLeaks( );
}

Incomplete output:
'Testing.exe' (Win32): Loaded 'C:\Users\so\source\repos\Testing\Debug\Testing.exe'. Symbols loaded.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ntdll.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\kernel32.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\KernelBase.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msvcp140d.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\vcruntime140d.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\ucrtbased.dll'.
The thread 0x4a8c has exited with code 0 (0x0).
Detected memory leaks!
Dumping objects ->
{20262} normal block at 0x013AF5E0, 4 bytes long.
Data: < > 00 00 00 00
{20242} normal block at 0x013AF190, 4 bytes long.
Data: < > 07 00 00 00
{20222} normal block at 0x013AF1F0, 4 bytes long.
Data: < > 07 00 00 00
{20201} normal block at 0x013AF580, 4 bytes long.
Data: < > 00 00 00 00
{20181} normal block at 0x013AF4C0, 4 bytes long.
Data: < > 03 00 00 00
{20161} normal block at 0x013AF460, 4 bytes long.
Data: < > 03 00 00 00
{20141} normal block at 0x013AF1C0, 4 bytes long.
Data: < > 00 00 00 00
{20121} normal block at 0x013AF310, 4 bytes long.
Data: < > 06 00 00 00
{20101} normal block at 0x013AF640, 4 bytes long.
Data: < > 01 00 00 00
{20081} normal block at 0x013AF220, 4 bytes long.
Data: < > 04 00 00 00
{20061} normal block at 0x013AF610, 4 bytes long.
Data: < > 08 00 00 00
{20041} normal block at 0x013AF490, 4 bytes long.
Data: < > 01 00 00 00
{20021} normal block at 0x013AF430, 4 bytes long.
Data: < > 00 00 00 00
{20001} normal block at 0x013AF160, 4 bytes long.
Data: < > 02 00 00 00
{19981} normal block at 0x013AF100, 4 bytes long.
Data: < > 08 00 00 00
{19960} normal block at 0x013AF0D0, 4 bytes long.
Data: < > 09 00 00 00
{19940} normal block at 0x013AF0A0, 4 bytes long.
Data: < > 08 00 00 00
{19919} normal block at 0x013AEE30, 4 bytes long.
Data: < > 09 00 00 00
{19899} normal block at 0x013AEF50, 4 bytes long.
Data: < > 01 00 00 00
{19879} normal block at 0x013AEE00, 4 bytes long.
Data: < > 09 00 00 00
{19859} normal block at 0x013AEDD0, 4 bytes long.
Data: < > 02 00 00 00
{19839} normal block at 0x013AEF80, 4 bytes long.
Data: < > 02 00 00 00
{19819} normal block at 0x013AF070, 4 bytes long.
Data: < > 06 00 00 00
{19799} normal block at 0x013AED40, 4 bytes long.
Data: < > 02 00 00 00
{19779} normal block at 0x013AF040, 4 bytes long.
Data: < > 09 00 00 00
{19759} normal block at 0x013AEF20, 4 bytes long.
Data: < > 01 00 00 00
{19739} normal block at 0x013AED70, 4 bytes long.
Data: < > 08 00 00 00
{19719} normal block at 0x013AEC50, 4 bytes long.
Data: < > 02 00 00 00
{19699} normal block at 0x013AEEC0, 4 bytes long.
Data: < > 06 00 00 00
{19679} normal block at 0x013AEC20, 4 bytes long.
Data: < > 02 00 00 00
{19659} normal block at 0x013AEFE0, 4 bytes long.
Data: < > 00 00 00 00
{19639} normal block at 0x013AEBF0, 4 bytes long.
Data: < > 02 00 00 00
{19619} normal block at 0x013AEB90, 4 bytes long.
Data: < > 09 00 00 00
{19598} normal block at 0x013AEDA0, 4 bytes long.
Data: < > 00 00 00 00
{19578} normal block at 0x013AEFB0, 4 bytes long.
Data: < > 04 00 00 00
{19558} normal block at 0x013AF010, 4 bytes long.
Data: < > 09 00 00 00
{19538} normal block at 0x013AEBC0, 4 bytes long.
Data: < > 04 00 00 00
{19518} normal block at 0x013AECE0, 4 bytes long.
Data: < > 02 00 00 00
{19498} normal block at 0x013AEE90, 4 bytes long.
Data: < > 02 00 00 00
{19478} normal block at 0x013AEE60, 4 bytes long.
Data: < > 00 00 00 00
{19458} normal block at 0x013AED10, 4 bytes long.
Data: < > 06 00 00 00
{19438} normal block at 0x013AEC80, 4 bytes long.
Data: < > 00 00 00 00
{19418} normal block at 0x013AEB60, 4 bytes long.
Data: < > 03 00 00 00
{19398} normal block at 0x013AEEF0, 4 bytes long.
Data: < > 03 00 00 00
{19378} normal block at 0x013AE830, 4 bytes long.
Data: < > 09 00 00 00
{19357} normal block at 0x013AECB0, 4 bytes long.
Data: < > 05 00 00 00
{19337} normal block at 0x013AE890, 4 bytes long.
Data: < > 07 00 00 00
{19317} normal block at 0x013AE800, 4 bytes long.
Data: < > 01 00 00 00
{19297} normal block at 0x013AE860, 4 bytes long.
Data: < > 02 00 00 00
{19277} normal block at 0x013AEA70, 4 bytes long.
Data: < > 01 00 00 00
{19257} normal block at 0x013AE9B0, 4 bytes long.
Data: < > 06 00 00 00
{19237} normal block at 0x013AE680, 4 bytes long.
Data: < > 05 00 00 00
{19217} normal block at 0x013AEA10, 4 bytes long.
Data: < > 09 00 00 00
{19197} normal block at 0x013AEAA0, 4 bytes long.
Data: < > 09 00 00 00
{19176} normal block at 0x013AE650, 4 bytes long.
Data: < > 00 00 00 00
{19156} normal block at 0x013AE950, 4 bytes long.
Data: < > 07 00 00 00
{19136} normal block at 0x013AE5F0, 4 bytes long.
Data: < > 07 00 00 00
{19115} normal block at 0x013AE620, 4 bytes long.
Data: < > 07 00 00 00
{19095} normal block at 0x013AEAD0, 4 bytes long.
Data: < > 03 00 00 00
{19075} normal block at 0x013AE6B0, 4 bytes long.
Data: < > 06 00 00 00
{19055} normal block at 0x013AE7D0, 4 bytes long.
Data: < > 03 00 00 00
{19035} normal block at 0x013AE7A0, 4 bytes long.
Data: < > 03 00 00 00
{19015} normal block at 0x013AE9E0, 4 bytes long.
Data: < > 02 00 00 00
{18995} normal block at 0x013AE6E0, 4 bytes long.
Data: < > 04 00 00 00
{18975} normal block at 0x013AE740, 4 bytes long.
Data: < > 02 00 00 00
{18955} normal block at 0x013AE920, 4 bytes long.
Data: < > 09 00 00 00
{18935} normal block at 0x013AE530, 4 bytes long.
Data: < > 02 00 00 00
{18915} normal block at 0x013AEA40, 4 bytes long.
Data: < > 03 00 00 00
{18895} normal block at 0x013AE770, 4 bytes long.
Data: < > 00 00 00 00
{18875} normal block at 0x013AE5C0, 4 bytes long.
Data: < > 04 00 00 00
{18855} normal block at 0x013AE980, 4 bytes long.
Data: < > 02 00 00 00
{18835} normal block at 0x013AE560, 4 bytes long.
Data: < > 05 00 00 00
{18815} normal block at 0x013AE710, 4 bytes long.
Data: < > 06 00 00 00
{18795} normal block at 0x013AE8F0, 4 bytes long.
Data: < > 07 00 00 00
{18774} normal block at 0x013AE8C0, 4 bytes long.
Data: < > 08 00 00 00
{18754} normal block at 0x013AEB00, 4 bytes long.
Data: < > 09 00 00 00
{18733} normal block at 0x013AE590, 4 bytes long.
Data: < > 01 00 00 00
{18713} normal block at 0x013AE0B0, 4 bytes long.
Data: < > 08 00 00 00
{18693} normal block at 0x013AE410, 4 bytes long.
Data: < > 06 00 00 00
{18673} normal block at 0x013AE3E0, 4 bytes long.
Data: < > 07 00 00 00
{18653} normal block at 0x013AE320, 4 bytes long.
Data: < > 03 00 00 00
{18633} normal block at 0x013AE350, 4 bytes long.
Data: < > 02 00 00 00
{18613} normal block at 0x013AE080, 4 bytes long.
Data: < > 04 00 00 00
{18593} normal block at 0x013AE4D0, 4 bytes long.
Data: < > 00 00 00 00
{18573} normal block at 0x013AE050, 4 bytes long.
Data: < > 00 00 00 00
{18553} normal block at 0x013AE260, 4 bytes long.
Data: < > 09 00 00 00
{18532} normal block at 0x013AE2C0, 4 bytes long.
Data: < > 02 00 00 00
{18512} normal block at 0x013AE1D0, 4 bytes long.
Data: < > 01 00 00 00
{18492} normal block at 0x013AE020, 4 bytes long.
Data: < > 05 00 00 00
{18472} normal block at 0x013ADFF0, 4 bytes long.
Data: < > 04 00 00 00
{18452} normal block at 0x013AE4A0, 4 bytes long.
Data: < > 03 00 00 00
{18432} normal block at 0x013AE470, 4 bytes long.
Data: < > 09 00 00 00
{18412} normal block at 0x013AE440, 4 bytes long.
Data: < > 05 00 00 00
{18392} normal block at 0x013AE230, 4 bytes long.
Data: < > 08 00 00 00
{18371} normal block at 0x013AE1A0, 4 bytes long.
Data: < > 07 00 00 00
{18351} normal block at 0x013AE170, 4 bytes long.
Data: < > 09 00 00 00
{18331} normal block at 0x013AE380, 4 bytes long.
Data: < > 04 00 00 00
{18311} normal block at 0x013ADF60, 4 bytes long.
Data: < > 05 00 00 00
{18291} normal block at 0x013AE500, 4 bytes long.
Data: < > 02 00 00 00
{18271} normal block at 0x013AE200, 4 bytes long.
Data: < > 01 00 00 00
{18251} normal block at 0x013AE2F0, 4 bytes long.
Data: < > 03 00 00 00
{18231} normal block at 0x013ADF90, 4 bytes long.
Data: < > 01 00 00 00
{18211} normal block at 0x013AE3B0, 4 bytes long.
Data: < > 02 00 00 00
{18191} normal block at 0x013AE290, 4 bytes long.
Data: < > 04 00 00 00
{18171} normal block at 0x013ADFC0, 4 bytes long.
Data: < > 02 00 00 00
{18151} normal block at 0x013AE140, 4 bytes long.
Data: < > 03 00 00 00
{18131} normal block at 0x013AE110, 4 bytes long.
Data: < > 05 00 00 00
{18111} normal block at 0x013ADF30, 4 bytes long.
Data: < > 05 00 00 00
{18091} normal block at 0x013AE0E0, 4 bytes long.
Data: < > 01 00 00 00
{18071} normal block at 0x013ADCC0, 4 bytes long.

.
.
.

Object dump complete.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\kernel.appcore.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\msvcrt.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\rpcrt4.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sspicli.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\cryptbase.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\bcryptprimitives.dll'.
'Testing.exe' (Win32): Loaded 'C:\Windows\SysWOW64\sechost.dll'.
The thread 0x5ddc has exited with code 0 (0x0).
The thread 0x5c68 has exited with code 0 (0x0).
The thread 0x5aac has exited with code 0 (0x0).
The program '[24520] Testing.exe' has exited with code 0 (0x0).

Pozdrav!

[Martinsos]

Thanks @SigmaOrionis!

"By all coding standards" -> I believe you are looking at this from the C++ perspective? I have to admit that I am no C/C++ expert, but I don't think that Edlib is going against any coding standards with the current API, which is really a C API, which enables it to be used both in C and C++. It is clearly stated in docs that the resulting object should be freed with the appropriate function.
With that said, I am open to suggestions that could improve Edlib -> what and how (in details) would you change in order to make Edlib API better in this case? Please keep in mind that is has to work both in C and in C++.

Regarding the memory leak, thank you for this great code example, this helps a lot!
Problem I am having though is that I don't have Windows environment ready to test this (I don't use Windows for programming) and I failed to reproduce the error in Linux. I used

#include <cstdlib>

#include <stdio.h>
#include <stdlib.h>     
#include <time.h>       
#include <string>
#include "edlib.h"

std::string getRandomString( )
{
    int randlen = rand( ) % 10 + 1; // rand between 1 and 10
    char randChar;
    std::string out;

    for( int i = 0; i < randlen; i++ )
    {
        randChar = rand( ) % 122 + 48;
        out.push_back( randChar );
    }

    return out;
}

int main()
{
    srand( time( nullptr ) );

    std::string one;
    std::string two;

    for( int i = 0; i < 1000; i++ )
    {
        one = getRandomString( );
        two = getRandomString( );

        auto ed = edlibAlign( one.c_str( ), one.length( ), two.c_str( ), two.length( ), edlibDefaultAlignConfig( ) ).editDistance;
    }
}

And then compiled and tested it with

c++ edlib-test.cpp edlib/src/edlib.cpp -o test -I edlib/include
valgrind --tool=memcheck -s --leak-check=full test

to check for leaks and I get

==9930== Memcheck, a memory error detector
==9930== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==9930== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==9930== Command: test
==9930== 
==9930== 
==9930== HEAP SUMMARY:
==9930==     in use at exit: 0 bytes in 0 blocks
==9930==   total heap usage: 30 allocs, 30 frees, 4,065 bytes allocated
==9930== 
==9930== All heap blocks were freed -- no leaks are possible
==9930== 
==9930== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

I unfortunately don't right now have time to set up Windows environment to reproduce exactly what you did.
Is this something you might be interested looking into, identifying possible cause of the memory leak, since you can already reproduce it easily? If you find a cause I could help you fix it and you could create a PR!

[SigmaOrionis]

Sorry Martin, tried to help here, have some other projects to work on.

Kind regards.

[Martinsos]

@SigmaOrionis np, thanks for help so far.

I will leave the issue open, in case somebody with Windows and some time on their hands would like to try to reproduce and possibly fix the possible bug!