Re: What does Substitute() actually do?

[Andy Seaborne <an...@apache.org>]

On 28/02/2025 23:25, Martynas Jusevičius wrote:
> OK, I realized only now that the ParameterizedSparqlString is safe in
> terms of injection attacks but not in terms of SPARQL scoping... Since
> it's just a wrapper for string replacement. I had mistaken this.
> 
> So I actually should be looking at the QueryTransformOps
> implementation probably.
> 
> And my question from the previous email should have been:
> Can there be differences in query semantics after a value was injected
> using *QueryTransformOps* vs. Substitute?

QueryTransformOps will replace variables inside sub-select regardless of 
scope. It is "parameterized query" use case.

https://afs.github.io/substitute.html (which is not the class 
"Substitute" in the code base) has a processing step before replacing 
variables that renames sub-select variables.

QueryTransformOps will have a variant that includes the replacement 
values as select expressions, and variant that does not.

     Andy

> On Wed, Feb 26, 2025 at 9:08 PM Martynas Jusevičius
> <ma...@atomgraph.com> wrote:
>>
>> On Fri, Feb 21, 2025 at 1:01 PM Andy Seaborne <an...@apache.org> wrote:
>>>
>>>
>>>
>>> On 12/02/2025 21:03, Martynas Jusevičius wrote:
>>>> Hi Andy,
>>>>
>>>> I understand the differences of how they are implemented.
>>>>
>>>> I think what I'm missing is a sort of overview of when those different
>>>> approaches should be used, and also examples of when they produce
>>>> semantically different queries.
>>>
>>> string template substitution can alter or break queries.
>>>
>>> Replace ?X in {?s :p ?X} with the  '"text" . FILTER (?s != :hideMe )'
>>
>> I understand that simple string replacement is not safe.
>>
>> But can there be differences in query semantics after a value was
>> injected using ParameterizedSparqlString vs. Substitute?
>>
>>>
>>> https://xkcd.com/327/
>>>
>>>       Andy
>>>
>>>>
>>>>
>>>> Martynas
>>>>
>>>> On Thu, Feb 6, 2025 at 5:59 PM Andy Seaborne <an...@apache.org> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 06/02/2025 03:24, Nicholas Car wrote:
>>>>>> You could try running that with RDFlib and see what it does….
>>>>>>
>>>>>> How is substitute() different to just string template substitution? Is it just a convenience function so substitution can be done on a Jena SPARQL object (perhaps algebra) rather than a string?
>>>>>
>>>>>
>>>>> It does not work on a string - when working with Query objects (e.g.
>>>>> service enhancer), manipulate is on Query objects.
>>>>> It does does not have injection attacks.
>>>>>
>>>>> There some places where you can't replace a variable by a value
>>>>>
>>>>> BIND(123  AS ?X)
>>>>>
>>>>> for example. QueryTransforOps checks.
>>>>>
>>>>> Substitute.substitue is a collect of algebra to algebra changes - some
>>>>> of which are used in execution.
>>>>>
>>>>>>
>>>>>> Is that your goal for an RDFLib equivalent?
>>>>>
>>>>> See also ParameterizedSparqlString in Jena which is injection-safe.
>>>>> That's closer to string replacement.
>>>>>
>>>>> Under the general concept of "substitute" there are different uses cases
>>>>> on different objects.
>>>>>
>>>>>        Andy
>>>>>
>>>>>> SPARQL string to/from SPARQL grammar object is handles in RDFLib fine so I guess as I think the code says, an RDFLib substitute() could work on either or both.
>>>>>>
>>>>>> Nick
>>>>>>
>>>>>> On Thu, Feb 6, 2025 at 10:05 am, Martynas Jusevičius <[martynas@atomgraph.com](mailto:On Thu, Feb 6, 2025 at 10:05 am, Martynas Jusevičius <<a href=)> wrote:
>>>>>>
>>>>>>> OK I simply fed Jena's source to ChatGPT and asked it to produce an
>>>>>>> RDFLIb version :)
>>>>>>> No idea if it's correct but I'll find out I guess.
>>>>>>>
>>>>>>> from rdflib import Variable, BNode, Literal
>>>>>>> from rdflib.plugins.sparql.algebra import BGP, Filter, Join, LeftJoin,
>>>>>>> Union, Graph, Extend, Minus, OrderBy, Project
>>>>>>>
>>>>>>> def substitute(algebra, binding):
>>>>>>> if isinstance(algebra, BGP):
>>>>>>> new_triples = []
>>>>>>> for s, p, o in algebra.triples:
>>>>>>> s = binding.get(s, s)
>>>>>>> p = binding.get(p, p)
>>>>>>> o = binding.get(o, o)
>>>>>>> new_triples.append((s, p, o))
>>>>>>> return BGP(new_triples)
>>>>>>> elif isinstance(algebra, Filter):
>>>>>>> new_p = substitute(algebra.p, binding)
>>>>>>> new_expr = algebra.expr
>>>>>>> for var, val in binding.items():
>>>>>>> new_expr = new_expr.substitute(var, val)
>>>>>>> return Filter(new_expr, new_p)
>>>>>>> elif isinstance(algebra, Join):
>>>>>>> return Join(substitute(algebra.p1, binding),
>>>>>>> substitute(algebra.p2, binding))
>>>>>>> elif isinstance(algebra, LeftJoin):
>>>>>>> return LeftJoin(substitute(algebra.p1, binding),
>>>>>>> substitute(algebra.p2, binding), algebra.expr)
>>>>>>> elif isinstance(algebra, Union):
>>>>>>> return Union(substitute(algebra.p1, binding),
>>>>>>> substitute(algebra.p2, binding))
>>>>>>> elif isinstance(algebra, Graph):
>>>>>>> return Graph(algebra.term, substitute(algebra.p, binding))
>>>>>>> elif isinstance(algebra, Extend):
>>>>>>> new_p = substitute(algebra.p, binding)
>>>>>>> new_expr = algebra.expr
>>>>>>> for var, val in binding.items():
>>>>>>> new_expr = new_expr.substitute(var, val)
>>>>>>> return Extend(new_p, new_expr, algebra.var)
>>>>>>> elif isinstance(algebra, Minus):
>>>>>>> return Minus(substitute(algebra.p1, binding),
>>>>>>> substitute(algebra.p2, binding))
>>>>>>> elif isinstance(algebra, OrderBy):
>>>>>>> return OrderBy(substitute(algebra.p, binding), algebra.expr)
>>>>>>> elif isinstance(algebra, Project):
>>>>>>> return Project(substitute(algebra.p, binding), algebra.PV)
>>>>>>> else:
>>>>>>> return algebra
>>>>>>>
>>>>>>> On Thu, Feb 6, 2025 at 12:54 AM Martynas Jusevičius
>>>>>>> <ma...@atomgraph.com> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I was looking to implement something like substitute() using RDFLib.
>>>>>>>>
>>>>>>>> I checked the note:
>>>>>>>> https://afs.github.io/substitute.htm
>>>>>>>> But I couldn't find a clear example.
>>>>>>>>
>>>>>>>> For example, how would this query string (the template)
>>>>>>>>
>>>>>>>> PREFIX dbo: <http://dbpedia.org/ontology/>
>>>>>>>> CONSTRUCT
>>>>>>>> WHERE { ?city dbo:populationTotal ?population }
>>>>>>>>
>>>>>>>> would look like after substituting (?city,
>>>>>>>> <http://dbpedia.org/resource/Copenhagen>)?
>>>>>>>>
>>>>>>>>
>>>>>>>> Martynas
>>>>>
>>>