You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2021/02/16 14:58:03 UTC

Standards, specs for URL parameter ordering

All,

I'm sorry for using users@ as my own personal Google but I'm sure 
someone knows this off the top of their head and can save me a lot of 
reading.

I'm wondering about which specs mention how to handle URL parameters 
(and POST parameters, for that matter) in terms of ordering. For 
example, if I have a URL like:

https://example.com/context/resource?a=1&b=2&c=3&a=6

(Note that I have "a" in there twice)

If I call request.getParameterNames(), is there a predictable order in 
which those parameters will be handed back? I'd love to hear that not 
only are they returned in "URL order" (that is, the left-most parameter 
is the first returned in that enumeration) in Tomcat, but either the 
servlet spec, the CGI spec, or some other spec dictates that order 
explicitly.

Similarly, if I use request.getParameterMap and than iterate through the 
keys or Map.Entry objects in it, does that behave predictably as well?

I have a situation where both URL parameters and POST parameters being 
in URL-order (or document-order for POST parameters) would be highly 
convenient, but I'd like to know if I can actually *rely* on that 
behavior, or if I have to make arrangements to provide the ordering in a 
separate parameter.

I'd like to know which specs mention these things if they are indeed 
specified in any of them.

Thanks in advance,
-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Standards, specs for URL parameter ordering

Posted by Konstantin Kolinko <kn...@gmail.com>.
ср, 17 февр. 2021 г. в 20:05, Christopher Schultz
<ch...@christopherschultz.net>:
>
> Konstantin,
>
> [...]
>
> Without being able to rely on form-data ordering, the only way to "know"
> which question was asked first, second, etc. would be to either put a
> hidden form element on the page describing the question-order (back to
> the server) or to have individual hidden elements for each question
> specifying the order. Something like this:
>
>    <input type="hidden" name="question_3762_order" value="1" />
>    <input type="hidden" name="question_213_order" value="2" />
>    ...

An alternative solution can be to include both question id and order
in the field name. E.g.

<input name="answer_1_3762" placeholder="Answer for Q3762">
<input name="answer_2_213" placeholder="Answer for Q213">

Well, your use case is a valid one.

BTW, for the same grounds for forms submitted with mime type of
"multipart/form-data"
the HttpServletRequest.getParts() method could return an ordered
collection (a List).
As of now, the API is specified to return a Collection<Part> [1].

[1] https://javaee.github.io/javaee-spec/javadocs/javax/servlet/http/HttpServletRequest.html#getParts--

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Standards, specs for URL parameter ordering

Posted by Christopher Schultz <ch...@christopherschultz.net>.
Konstantin,

On 2/17/21 10:35, Konstantin Kolinko wrote:
> ср, 17 февр. 2021 г. в 12:09, Mark Thomas <ma...@apache.org>:
>>
>> On 16/02/2021 14:58, Christopher Schultz wrote:
>>> All,
>>>
>>> I'm sorry for using users@ as my own personal Google but I'm sure
>>> someone knows this off the top of their head and can save me a lot of
>>> reading.
>>>
>>> I'm wondering about which specs mention how to handle URL parameters
>>> (and POST parameters, for that matter) in terms of ordering. For
>>> example, if I have a URL like:
>>>
>>> https://example.com/context/resource?a=1&b=2&c=3&a=6
>>>
>>> (Note that I have "a" in there twice)
>>>
>>> If I call request.getParameterNames(), is there a predictable order in
>>> which those parameters will be handed back? I'd love to hear that not
>>> only are they returned in "URL order" (that is, the left-most parameter
>>> is the first returned in that enumeration) in Tomcat, but either the
>>> servlet spec, the CGI spec, or some other spec dictates that order
>>> explicitly.
>>
>> Yes, they will be in that order. (See ApplicationHttpRequest.parameters,
>> ParameterMap.delgatedMap and LinkedHashMap
>>
>> The order isn't explicitly defined in any specification I am aware of.
>> However, the Servlet spec does state (3.1) that query string parameters
>> should be presented before parameters parsed form the request body.
> 
> 1. When there are multiple values, the order of values is preserved.
> 
> Java Servlet 4.0 spec has an example in its text that shows how the
> order of values is preserved (chapter 3.1 "HTTP Protocol Parameters"):
> 
> <quote>Data from the query string and the post body are aggregated
> into the request parameter set. Query string data is presented before
> post body data. For example, if a request is made with a query string
> of a=hello and a post body of a=goodbye&a=world, the resulting
> parameter set would be ordered a=(hello, goodbye, world)</quote>

Yes, that's the section Mark was referencing. While the example may be 
instructive, it is not explicit. I'm just requesting explicit language 
in the spec for that assumption.

> 2. Original specification for url-encoded parameters is not HTTP, but
> HTML specification. The place where I first saw it many years ago was
> here:
> 
> https://tools.ietf.org/html/rfc1866#section-8.2.1
> HTML 2.0 spec
> 
>> The fields are listed in the order they appear in the document

+1

> Personally, I would not rely on the order of names provided by
> getParameterNames(), as it would be surprising for a client if
> processing of a request depends on the order of parameter names. In
> the same way as whether reordering of input fields on a web form could
> break its processing.

I disagree: both HTML and HTTP make it clear that form-data ordering is 
important and should be preserved. Given the importance placed on that 
ordering, I think that same importance and reliable ordering should be 
extended explicitly into the Servlet Specification explicitly.

In my use-case, I have a series of questions being presented on the 
screen. When I receive the answers to those questions, I'd like to be 
able to rely on the form-data ordering to know which questions were 
presented first and which ones later when they are all on the same form.

Without being able to rely on form-data ordering, the only way to "know" 
which question was asked first, second, etc. would be to either put a 
hidden form element on the page describing the question-order (back to 
the server) or to have individual hidden elements for each question 
specifying the order. Something like this:

   <input type="hidden" name="question_3762_order" value="1" />
   <input type="hidden" name="question_213_order" value="2" />
   ...

If the ordering can be reliable, less data needs to be exchanged.

I have presented a justification for this in the issue filed for the 
Servlet API[1] and I'm happy to hear any critique you may have of that 
justification.

For emphasis:
> Personally, I would not rely on the order of names provided by 
> getParameterNames()
It is precisely because of the ambiguity of the specification that I am 
requesting clarification and, further, explicit definitive required 
behavior of the Servlet Specification such that application developers 
*can* rely on that ordering.

-chris

[1] https://github.com/eclipse-ee4j/servlet-api/issues/393

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Standards, specs for URL parameter ordering

Posted by Konstantin Kolinko <kn...@gmail.com>.
ср, 17 февр. 2021 г. в 12:09, Mark Thomas <ma...@apache.org>:
>
> On 16/02/2021 14:58, Christopher Schultz wrote:
> > All,
> >
> > I'm sorry for using users@ as my own personal Google but I'm sure
> > someone knows this off the top of their head and can save me a lot of
> > reading.
> >
> > I'm wondering about which specs mention how to handle URL parameters
> > (and POST parameters, for that matter) in terms of ordering. For
> > example, if I have a URL like:
> >
> > https://example.com/context/resource?a=1&b=2&c=3&a=6
> >
> > (Note that I have "a" in there twice)
> >
> > If I call request.getParameterNames(), is there a predictable order in
> > which those parameters will be handed back? I'd love to hear that not
> > only are they returned in "URL order" (that is, the left-most parameter
> > is the first returned in that enumeration) in Tomcat, but either the
> > servlet spec, the CGI spec, or some other spec dictates that order
> > explicitly.
>
> Yes, they will be in that order. (See ApplicationHttpRequest.parameters,
> ParameterMap.delgatedMap and LinkedHashMap
>
> The order isn't explicitly defined in any specification I am aware of.
> However, the Servlet spec does state (3.1) that query string parameters
> should be presented before parameters parsed form the request body.

1. When there are multiple values, the order of values is preserved.

Java Servlet 4.0 spec has an example in its text that shows how the
order of values is preserved (chapter 3.1 "HTTP Protocol Parameters"):

<quote>Data from the query string and the post body are aggregated
into the request parameter set. Query string data is presented before
post body data. For example, if a request is made with a query string
of a=hello and a post body of a=goodbye&a=world, the resulting
parameter set would be ordered a=(hello, goodbye, world)</quote>

2. Original specification for url-encoded parameters is not HTTP, but
HTML specification. The place where I first saw it many years ago was
here:

https://tools.ietf.org/html/rfc1866#section-8.2.1
HTML 2.0 spec

> The fields are listed in the order they appear in the document

Personally, I would not rely on the order of names provided by
getParameterNames(), as it would be surprising for a client if
processing of a request depends on the order of parameter names. In
the same way as whether reordering of input fields on a web form could
break its processing.

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Standards, specs for URL parameter ordering

Posted by Christopher Schultz <ch...@christopherschultz.net>.
All,

On 2/17/21 09:22, Mark Thomas wrote:
> On 17/02/2021 14:03, Christopher Schultz wrote:
>> Mark,
>>
>> On 2/17/21 04:08, Mark Thomas wrote:
>>> On 16/02/2021 14:58, Christopher Schultz wrote:
>>>> All,
>>>>
>>>> I'm sorry for using users@ as my own personal Google but I'm sure
>>>> someone knows this off the top of their head and can save me a lot of
>>>> reading.
>>>>
>>>> I'm wondering about which specs mention how to handle URL parameters
>>>> (and POST parameters, for that matter) in terms of ordering. For
>>>> example, if I have a URL like:
>>>>
>>>> https://example.com/context/resource?a=1&b=2&c=3&a=6
>>>>
>>>> (Note that I have "a" in there twice)
>>>>
>>>> If I call request.getParameterNames(), is there a predictable order in
>>>> which those parameters will be handed back? I'd love to hear that not
>>>> only are they returned in "URL order" (that is, the left-most parameter
>>>> is the first returned in that enumeration) in Tomcat, but either the
>>>> servlet spec, the CGI spec, or some other spec dictates that order
>>>> explicitly.
>>>
>>> Yes, they will be in that order. (See ApplicationHttpRequest.parameters,
>>> ParameterMap.delgatedMap and LinkedHashMap
>>>
>>> The order isn't explicitly defined in any specification I am aware of.
>>> However, the Servlet spec does state (3.1) that query string parameters
>>> should be presented before parameters parsed form the request body.
>>> There are several ways to implement that but it is likely that an
>>> implementation will simply maintain insertion order for keys and values
>>> (where there are multiple values for the same key) in a single
>>> collection. At least, Tomcat does this.
>>>
>>>> Similarly, if I use request.getParameterMap and than iterate through the
>>>> keys or Map.Entry objects in it, does that behave predictably as well?
>>>
>>> Same answer as above.
>>>
>>>> I have a situation where both URL parameters and POST parameters being
>>>> in URL-order (or document-order for POST parameters) would be highly
>>>> convenient, but I'd like to know if I can actually *rely* on that
>>>> behavior, or if I have to make arrangements to provide the ordering in a
>>>> separate parameter.
>>>
>>> You can rely on this if you are using Tomcat. I don't see the underlying
>>> design changing anytime soon. For other containers, YMMV.
>>>
>>>> I'd like to know which specs mention these things if they are indeed
>>>> specified in any of them.
>>>
>>> The only thing I am aware of is the query string before request body
>>> ordering in section 3.1. of the Servlet spec.
>>
>> Thanks for all that. Experiments with Tomcat showed that the parameter
>> order was what I was expecting. I was wondering if that was simply an
>> implementation detail or if it was spec-mandated. It seems it's an
>> implementation detail, though it may be a popular one.
>>
>> I wonder how difficult it would be to lobby the Servlet EG to codify
>> that behavior in the specification. I'm sure it would be advantageous to
>> all application developers if they could predict the order of
>> parameter-processing on the server side given the structure of their
>> URLs and HTTP POST form data.
>>
>> My past attempts to get Servlet EG to do anything have been fruitless,
>> even for simple things[1]. Any idea if this kind of request would ever
>> actually be considered?
> 
> I don't see why not. Open an issue and we'll see.
> 
>> -chris
>>
>> [1] https://github.com/eclipse-ee4j/servlet-api/issues/130
> 
> Things should start improving now we have reached the point in Jakarta
> EE where we can evolve the specs and add new functionality. I hopeful
> for a LOT of additional clarity and a fair number of small new features
> in servlet.next.

Filed: https://github.com/eclipse-ee4j/servlet-api/issues/393

Any support from the Tomcat community would be greatly appreciated.

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Standards, specs for URL parameter ordering

Posted by Mark Thomas <ma...@apache.org>.
On 17/02/2021 14:03, Christopher Schultz wrote:
> Mark,
> 
> On 2/17/21 04:08, Mark Thomas wrote:
>> On 16/02/2021 14:58, Christopher Schultz wrote:
>>> All,
>>>
>>> I'm sorry for using users@ as my own personal Google but I'm sure
>>> someone knows this off the top of their head and can save me a lot of
>>> reading.
>>>
>>> I'm wondering about which specs mention how to handle URL parameters
>>> (and POST parameters, for that matter) in terms of ordering. For
>>> example, if I have a URL like:
>>>
>>> https://example.com/context/resource?a=1&b=2&c=3&a=6
>>>
>>> (Note that I have "a" in there twice)
>>>
>>> If I call request.getParameterNames(), is there a predictable order in
>>> which those parameters will be handed back? I'd love to hear that not
>>> only are they returned in "URL order" (that is, the left-most parameter
>>> is the first returned in that enumeration) in Tomcat, but either the
>>> servlet spec, the CGI spec, or some other spec dictates that order
>>> explicitly.
>>
>> Yes, they will be in that order. (See ApplicationHttpRequest.parameters,
>> ParameterMap.delgatedMap and LinkedHashMap
>>
>> The order isn't explicitly defined in any specification I am aware of.
>> However, the Servlet spec does state (3.1) that query string parameters
>> should be presented before parameters parsed form the request body.
>> There are several ways to implement that but it is likely that an
>> implementation will simply maintain insertion order for keys and values
>> (where there are multiple values for the same key) in a single
>> collection. At least, Tomcat does this.
>>
>>> Similarly, if I use request.getParameterMap and than iterate through the
>>> keys or Map.Entry objects in it, does that behave predictably as well?
>>
>> Same answer as above.
>>
>>> I have a situation where both URL parameters and POST parameters being
>>> in URL-order (or document-order for POST parameters) would be highly
>>> convenient, but I'd like to know if I can actually *rely* on that
>>> behavior, or if I have to make arrangements to provide the ordering in a
>>> separate parameter.
>>
>> You can rely on this if you are using Tomcat. I don't see the underlying
>> design changing anytime soon. For other containers, YMMV.
>>
>>> I'd like to know which specs mention these things if they are indeed
>>> specified in any of them.
>>
>> The only thing I am aware of is the query string before request body
>> ordering in section 3.1. of the Servlet spec.
> 
> Thanks for all that. Experiments with Tomcat showed that the parameter
> order was what I was expecting. I was wondering if that was simply an
> implementation detail or if it was spec-mandated. It seems it's an
> implementation detail, though it may be a popular one.
> 
> I wonder how difficult it would be to lobby the Servlet EG to codify
> that behavior in the specification. I'm sure it would be advantageous to
> all application developers if they could predict the order of
> parameter-processing on the server side given the structure of their
> URLs and HTTP POST form data.
> 
> My past attempts to get Servlet EG to do anything have been fruitless,
> even for simple things[1]. Any idea if this kind of request would ever
> actually be considered?

I don't see why not. Open an issue and we'll see.

> -chris
> 
> [1] https://github.com/eclipse-ee4j/servlet-api/issues/130

Things should start improving now we have reached the point in Jakarta
EE where we can evolve the specs and add new functionality. I hopeful
for a LOT of additional clarity and a fair number of small new features
in servlet.next.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Standards, specs for URL parameter ordering

Posted by Christopher Schultz <ch...@christopherschultz.net>.
Mark,

On 2/17/21 04:08, Mark Thomas wrote:
> On 16/02/2021 14:58, Christopher Schultz wrote:
>> All,
>>
>> I'm sorry for using users@ as my own personal Google but I'm sure
>> someone knows this off the top of their head and can save me a lot of
>> reading.
>>
>> I'm wondering about which specs mention how to handle URL parameters
>> (and POST parameters, for that matter) in terms of ordering. For
>> example, if I have a URL like:
>>
>> https://example.com/context/resource?a=1&b=2&c=3&a=6
>>
>> (Note that I have "a" in there twice)
>>
>> If I call request.getParameterNames(), is there a predictable order in
>> which those parameters will be handed back? I'd love to hear that not
>> only are they returned in "URL order" (that is, the left-most parameter
>> is the first returned in that enumeration) in Tomcat, but either the
>> servlet spec, the CGI spec, or some other spec dictates that order
>> explicitly.
> 
> Yes, they will be in that order. (See ApplicationHttpRequest.parameters,
> ParameterMap.delgatedMap and LinkedHashMap
> 
> The order isn't explicitly defined in any specification I am aware of.
> However, the Servlet spec does state (3.1) that query string parameters
> should be presented before parameters parsed form the request body.
> There are several ways to implement that but it is likely that an
> implementation will simply maintain insertion order for keys and values
> (where there are multiple values for the same key) in a single
> collection. At least, Tomcat does this.
> 
>> Similarly, if I use request.getParameterMap and than iterate through the
>> keys or Map.Entry objects in it, does that behave predictably as well?
> 
> Same answer as above.
> 
>> I have a situation where both URL parameters and POST parameters being
>> in URL-order (or document-order for POST parameters) would be highly
>> convenient, but I'd like to know if I can actually *rely* on that
>> behavior, or if I have to make arrangements to provide the ordering in a
>> separate parameter.
> 
> You can rely on this if you are using Tomcat. I don't see the underlying
> design changing anytime soon. For other containers, YMMV.
> 
>> I'd like to know which specs mention these things if they are indeed
>> specified in any of them.
> 
> The only thing I am aware of is the query string before request body
> ordering in section 3.1. of the Servlet spec.

Thanks for all that. Experiments with Tomcat showed that the parameter 
order was what I was expecting. I was wondering if that was simply an 
implementation detail or if it was spec-mandated. It seems it's an 
implementation detail, though it may be a popular one.

I wonder how difficult it would be to lobby the Servlet EG to codify 
that behavior in the specification. I'm sure it would be advantageous to 
all application developers if they could predict the order of 
parameter-processing on the server side given the structure of their 
URLs and HTTP POST form data.

My past attempts to get Servlet EG to do anything have been fruitless, 
even for simple things[1]. Any idea if this kind of request would ever 
actually be considered?

-chris

[1] https://github.com/eclipse-ee4j/servlet-api/issues/130

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Standards, specs for URL parameter ordering

Posted by Mark Thomas <ma...@apache.org>.
On 16/02/2021 14:58, Christopher Schultz wrote:
> All,
> 
> I'm sorry for using users@ as my own personal Google but I'm sure
> someone knows this off the top of their head and can save me a lot of
> reading.
> 
> I'm wondering about which specs mention how to handle URL parameters
> (and POST parameters, for that matter) in terms of ordering. For
> example, if I have a URL like:
> 
> https://example.com/context/resource?a=1&b=2&c=3&a=6
> 
> (Note that I have "a" in there twice)
> 
> If I call request.getParameterNames(), is there a predictable order in
> which those parameters will be handed back? I'd love to hear that not
> only are they returned in "URL order" (that is, the left-most parameter
> is the first returned in that enumeration) in Tomcat, but either the
> servlet spec, the CGI spec, or some other spec dictates that order
> explicitly.

Yes, they will be in that order. (See ApplicationHttpRequest.parameters,
ParameterMap.delgatedMap and LinkedHashMap

The order isn't explicitly defined in any specification I am aware of.
However, the Servlet spec does state (3.1) that query string parameters
should be presented before parameters parsed form the request body.
There are several ways to implement that but it is likely that an
implementation will simply maintain insertion order for keys and values
(where there are multiple values for the same key) in a single
collection. At least, Tomcat does this.

> Similarly, if I use request.getParameterMap and than iterate through the
> keys or Map.Entry objects in it, does that behave predictably as well?

Same answer as above.

> I have a situation where both URL parameters and POST parameters being
> in URL-order (or document-order for POST parameters) would be highly
> convenient, but I'd like to know if I can actually *rely* on that
> behavior, or if I have to make arrangements to provide the ordering in a
> separate parameter.

You can rely on this if you are using Tomcat. I don't see the underlying
design changing anytime soon. For other containers, YMMV.

> I'd like to know which specs mention these things if they are indeed
> specified in any of them.

The only thing I am aware of is the query string before request body
ordering in section 3.1. of the Servlet spec.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org