Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix memory leak in lmdb.cc + improve performance while reading collections in resolveMultiMatches #2520

Open
wants to merge 4 commits into
base: v3/master
Choose a base branch
from
Open

Fix memory leak in lmdb.cc + improve performance while reading collections in resolveMultiMatches #2520

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
064b626
Remove memory leak in lmdb.cc while passing data from MDB_val to Vari…
ziollek Feb 24, 2021
f1c5f6b
Optimize fetching collection in resolveMultiMatches
ziollek Feb 24, 2021
440e3c9
Fix static-check violations
ziollek Feb 24, 2021
d980bef
Merge branch 'SpiderLabs:v3/master' into v3/master
ziollek Mar 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions headers/modsecurity/variable_value.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@ class VariableValue {
m_value(value != nullptr?*value:"")
{ }

VariableValue(std::string&& key,
std::string&& value)
: m_collection(""),
m_key(std::move(key)),
m_value(std::move(value))
{ m_keyWithCollection = m_key; }

VariableValue(const std::string *collection,
const std::string *key,
const std::string *value)
Expand All @@ -56,6 +63,14 @@ class VariableValue {
m_value(*value)
{ }

VariableValue(const std::string *collection,
std::string&& key,
std::string&& value)
: m_collection(*collection),
m_key(std::move(key)),
m_value(std::move(value))
{ m_keyWithCollection = m_collection + ":" + m_key; }

explicit VariableValue(const VariableValue *o) :
m_collection(o->m_collection),
m_key(o->m_key),
Expand Down
35 changes: 21 additions & 14 deletions src/collection/backend/lmdb.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,7 @@ void LMDB::resolveSingleMatch(const std::string& var,
reinterpret_cast<char *>(mdb_value_ret.mv_data),
mdb_value_ret.mv_size);
VariableValue *v = new VariableValue(&var, a);
delete a;
l->push_back(v);
}

Expand Down Expand Up @@ -498,22 +499,28 @@ void LMDB::resolveMultiMatches(const std::string& var,
while ((rc = mdb_cursor_get(cursor, &key, &data, MDB_NEXT)) == 0) {
l->insert(l->begin(), new VariableValue(
&m_name,
new std::string(reinterpret_cast<char *>(key.mv_data),
std::string(reinterpret_cast<char *>(key.mv_data),
key.mv_size),
new std::string(reinterpret_cast<char *>(data.mv_data),
std::string(reinterpret_cast<char *>(data.mv_data),
data.mv_size)));
}
} else {
while ((rc = mdb_cursor_get(cursor, &key, &data, MDB_NEXT)) == 0) {
char *a = reinterpret_cast<char *>(key.mv_data);
if (strncmp(var.c_str(), a, keySize) == 0) {
l->insert(l->begin(), new VariableValue(
&m_name,
new std::string(reinterpret_cast<char *>(key.mv_data),
key.mv_size),
new std::string(reinterpret_cast<char *>(data.mv_data),
data.mv_size)));
}
string2val(var, &key);
rc = mdb_cursor_get(cursor, &key, &data, MDB_SET_RANGE);
Copy link

@sobczak-m sobczak-m Mar 24, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ziollek replace MDB_SET_RANGE to MDB_SET_KEY
MDB_SET_RANGE - getting first key greater than or equal to specified key http://www.lmdb.tech/doc/group__mdb.html#ga1206b2af8b95e7f6b0ef6b28708c9127
If you are looking for none exist samplekey you get key containing samplekey like samplekey2 (if such key exist)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workable link for @sobczak-m comment's - http://www.lmdb.tech/doc/group__mdb.html#gga1206b2af8b95e7f6b0ef6b28708c9127af9feb0557c2954dbf7732eee5e1b59e7

if (rc == 0) {
do {
char *a = reinterpret_cast<char *>(key.mv_data);
if (strncmp(var.c_str(), a, keySize) == 0) {
l->insert(l->begin(), new VariableValue(
&m_name,
std::string(reinterpret_cast<char *>(key.mv_data),
key.mv_size),
std::string(reinterpret_cast<char *>(data.mv_data),
data.mv_size)));
} else {
break;
}
} while ((rc = mdb_cursor_get(cursor, &key, &data, MDB_NEXT)) == 0);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace MDB_NEXT to MDB_NEXT_DUB to iterate only over current key

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MDB_NEXT_DUP -
http://www.lmdb.tech/doc/group__mdb.html#gga1206b2af8b95e7f6b0ef6b28708c9127a37d22eecd320d77fbba9f905483f99e5

}
}

Expand Down Expand Up @@ -569,9 +576,9 @@ void LMDB::resolveRegularExpression(const std::string& var,
}

VariableValue *v = new VariableValue(
new std::string(reinterpret_cast<char *>(key.mv_data),
std::string(reinterpret_cast<char *>(key.mv_data),
key.mv_size),
new std::string(reinterpret_cast<char *>(data.mv_data),
std::string(reinterpret_cast<char *>(data.mv_data),
data.mv_size));
l->insert(l->begin(), v);
}
Expand Down