You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@jmeter.apache.org by bu...@apache.org on 2022/01/09 19:45:48 UTC
[Bug 65794] New: JSON Assertion always successful with filter operators
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Bug ID: 65794
Summary: JSON Assertion always successful with filter operators
Product: JMeter
Version: 5.4.3
Hardware: PC
Status: NEW
Severity: major
Priority: P2
Component: Main
Assignee: issues@jmeter.apache.org
Reporter: rosuvladimir@gmail.com
Target Milestone: JMETER_5.5
I'm testing an HTTP endpoint which returns the following content:
{
"isCompressed": true,
"rows": [
{
"_2": "",
"_4": "CustomString_1401",
"_5": {
"latitude": 0.0,
"longitude": 0.0,
"elevation": 0.0
},
"_6": "CustomString_1401",
"_8": "User0",
"_9": "User",
"_10": [],
"_11": 1641561103991
},
{
"_2": "",
"_4": "CustomString_3930",
"_5": {
"latitude": 0.0,
"longitude": 0.0,
"elevation": 0.0
},
"_6": "CustomString_3930",
"_8": "User0",
"_9": "User",
"_10": [],
"_11": 1641561045657
}
]
}
The following JSON assertion paths always return successful, even if those
paths do not exist.
$.rows[?(@._4=="Custom")]._4
$.rows[?(@._4=="Custom_3930")]._122
I believe this is a bug, as in case of assertion 1, there's no element which
satisfies the filter, and in case of Filter 2, there's no key whose name is
"_122".
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Vladimir Rosu <ro...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #4 from Vladimir Rosu <ro...@gmail.com> ---
I agree that with this patch the user would get the required feedback.
Two questions here.
1. Should I test a night build before releasing into mainstream? If so, do I
need to build it?
2. I would still add this note in the Help Center:
"When using indefinite JSON Paths you must assert the value due to the existing
JSON library implementation, otherwise the assertion could always return
successful"
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
--- Comment #13 from Vladimir Rosu <ro...@gmail.com> ---
Forgot to add that I tested with trunk 5.5-snapshot 85580b4
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Felix Schumacher <fe...@internetallee.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
--- Comment #10 from Felix Schumacher <fe...@internetallee.de> ---
*ping* @Vladimir, have you had time to check the changes?
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Vladimir Rosu <ro...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #8 from Vladimir Rosu <ro...@gmail.com> ---
I changed to New, because I'm not sure if I need to provide any other info.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
--- Comment #12 from Vladimir Rosu <ro...@gmail.com> ---
I confirm it works as discussed above. 100% ok from my PoV for release.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Felix Schumacher <fe...@internetallee.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
OS| |All
--- Comment #1 from Felix Schumacher <fe...@internetallee.de> ---
I think you are stumbling upon an implementation detail of the jsonpath library
we are using. According to
https://github.com/json-path/JsonPath#what-is-returned-when the library will
always return a list, if an expression '?(<expression>)' is used. JMeter will
interpret this as an empty (but valid) result.
We might be able to tweak the parser using the options given at
https://github.com/json-path/JsonPath#tweaking-configuration
Currently we are using the default configuration.
(./src/components/src/main/java/org/apache/jmeter/assertions/JSONPathAssertion.java
line 112)
You might get better results (without patching JMeter), if you were asserting a
value with the used assertion.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
--- Comment #6 from Felix Schumacher <fe...@internetallee.de> ---
The documentation note has been added.
commit 6a5cac0ad75e72d5313f3c795845c6cea9ff6520
AuthorDate: Tue Jan 18 17:36:49 2022 +0100
JSON Assertion always successful with indefinite paths
Add a note to the component reference for JSON Path Assertions
Bugzilla Id: 65794
---
xdocs/changes.xml | 2 ++
xdocs/usermanual/component_reference.xml | 3 +++
2 files changed, 5 insertions(+)
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
--- Comment #7 from Vladimir Rosu <ro...@gmail.com> ---
To save time, considering the patch is very low complexity, I do not need to
text. Absolutely ok with the approach.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Felix Schumacher <fe...@internetallee.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
Keywords| |PatchAvailable
--- Comment #5 from Felix Schumacher <fe...@internetallee.de> ---
The patch has not been applied yet. If you want to test it, you would have to
apply it by yourself and compile a version on your own. (If you feel, that you
are not up to the task, I could provide a compiled one)
If no other dev speaks against applying this patch, I will commit it.
The docs should be updated in any case.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Felix Schumacher <fe...@internetallee.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords| |FixedInTrunk
Status|NEW |NEEDINFO
--- Comment #9 from Felix Schumacher <fe...@internetallee.de> ---
@Vladimir, it would be great, if you could test next nightly or build from
trunk and report back, whether this fixes your problem.
commit 278255de85829c850d36ad1655d1541e66d5368f
AuthorDate: Sat Jan 15 18:11:36 2022 +0100
Fail JSONPath Assertion on indefinite paths and no assertion value
The docs for JSONPath Assertion state, that it will fail the assertion,
when no element is found with the given JSON path. This was currently
not followed, when an indefinite path was used. In such a case, the
JSONPath library would return an empty list, which the assertion logic
would log as OK.
With this change we let the assertion fail, when
* an indefinite path was given
* and an empty list is extracted
* and no assertion value is given
Bugzilla Id: 65794
---
.../java/org/apache/jmeter/assertions/JSONPathAssertion.java | 10 ++++++++--
.../org/apache/jmeter/assertions/TestJSONPathAssertion.java | 6 +++++-
xdocs/changes.xml | 2 +-
xdocs/usermanual/component_reference.xml | 4 +++-
4 files changed, 17 insertions(+), 5 deletions(-)
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
--- Comment #11 from Vladimir Rosu <ro...@gmail.com> ---
I will confirm tomorrow - sorry about the delay.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
--- Comment #3 from Felix Schumacher <fe...@internetallee.de> ---
Created attachment 38165
--> https://bz.apache.org/bugzilla/attachment.cgi?id=38165&action=edit
Break on indefinite paths, when an empty array is extracted and no assertion
value given
It would be great, if you would try to formulate the help text in a more
correct way.
Attached is a patch, that would make the assertion fail, if
* the given JSON Path is *indefinite*
* and an empty array is extracted
* and no assertion value has been given
I think it should fit with the current description better than silently
ignoring this case.
What do you think?
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Vladimir Rosu <ro...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #2 from Vladimir Rosu <ro...@gmail.com> ---
I confirm that the asserting value approach work, but in my situation I did not
want and need to generate the additional load of asserting the value.
After browsing to those links, I observed that a path, in JSON Path
terminology, can be indefinite in multiple situations, not only when using
Filter Operators.
Hence this situation might be triggered by other factors.
Considering the strange behavior I would like to add some helper text in the
JSON assertion Help Center, here:
https://jmeter.apache.org/usermanual/component_reference.html#JSON_Assertion
Is this solution acceptable? If yes, should I provide the text?
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 65794] JSON Assertion always successful with filter operators
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65794
Felix Schumacher <fe...@internetallee.de> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |FIXED
Status|NEEDINFO |RESOLVED
--
You are receiving this mail because:
You are the assignee for the bug.