You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by "Klein, Stephanie [USA]" <kl...@bah.com> on 2009/04/20 19:03:16 UTC
Can servers "block" programmatic form submissions?
Hi, All,
I'm writing an application to log into a website via a form. I've tried
many different ways, but each time, I get re-directed to a page saying
I'm being "redirected to an authentication service on another device".
(I'm not able to get to this page via a browser.)
I downloaded a browser sniffer program, but when it's running, the
website I'm trying to log into won't load properly (via the browser).
So, is it possible for a company to disable programmatic access on the
server side? Can they have something set up to block this type of
log-in?
Just in case it's helpful, here is the latest version of code I'm using.
Again, I've tried several versions. I'd be happy to share the other
versions if that will be helpful. I do have wire logging turned on and
would be happy to share those files also.
// BEGIN OF CODE
String response = "";
GetMethod authget = new GetMethod("https://www.somesite.com");
(new HttpClient()).executeMethod(authget);
response = authget.getResponseBodyAsString();
System.out.println("Body from authget " + response);
PostMethod authPost = new PostMethod("https://www.somesite.com");
NameValuePair[] paramList = {
new NameValuePair("SMENC", "ISO-8859-1"),
new NameValuePair("SMLOCAE", "US-EN"),
new NameValuePair("TARGET", "/login/index.html"),
new NameValuePair("USER", "username"),
new NameValuePair("PASSWORD", "password"),
new NameValuePair("lowBandwidth", "false")
};
System.out.println("Response body is " + response);
System.out.println("Login form post (authPost): " +
authPost.getStatusLine().toString());
// END OF CODE
Thank you for your time and consideration,
- Stephanie
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Can servers "block" programmatic form submissions?
Posted by mnenchev <ne...@gmail.com>.
sorry just testing
--
View this message in context: http://www.nabble.com/Can-servers-%22block%22-programmatic-form-submissions--tp23140751p23157972.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Can servers "block" programmatic form submissions?
Posted by mnenchev <ne...@gmail.com>.
Hi, i am new with this mailing lists stuff. Excuse me for spaming this topic,
but i dont know what else to do. I am subscribed fro this mail list(i
receive mails and i can post in other threads, but i can't post my thread
:(. I tried many things, i wrote to the support, nothing....
I am getting this error: mail delivery failed:returning message to sender
his message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
httpclient-users@hc.apache.org
SMTP error from remote mail server after end of data:
host mx1.us.apache.org [140.211.11.136]: 552 spam score (6.4) exceeded
threshold
...
What i am doing wrong?
Please help me, and excuse me for spaming this thread.
--
View this message in context: http://www.nabble.com/Can-servers-%22block%22-programmatic-form-submissions--tp23140751p23158265.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Can servers "block" programmatic form submissions?
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Tue, Apr 21, 2009 at 09:09:32AM -0400, Klein, Stephanie [USA] wrote:
> First, thank you for taking the time to respond to my question. I
> really appreciate it.
>
> "> I downloaded a browser sniffer program, but when it's running, the
> > website I'm trying to log into won't load properly (via the browser).
> >
>
> I find this highly improbable. I can not think of a way for a server
> side application to tell that the IP traffic is being sniffed upon, but
> I am not a network specialist by any stretch of imagination. There must
> be something else."
>
> I was rather surprised by the behavior of the website when I had the
> sniffer running, too. I tried several times, though - when the sniffer
> was running, the page wouldn't load. When the sniffer was stopped, the
> page loaded fine. I suppose it could be something with the firewall
> here at work - I'll try at home on my personal computer and see if I get
> the same results.
>
Were you using an HTTP sniffer that acted as a proxy to the target host or a full-flown protocol analyzer such as Wireshark?
> " there should always be a way to emulate the HTTP packets as emitted by
> the common browsers."
>
> Can you suggest a particular object I should research/explore to try to
> find a way to emulate the browser? Will I want to look at HttpClient,
> or is there something else I should look into?
>
There is no way around capturing HTTP packets generated by a browser and emulating the same HTTP session using HttpClient (or some other HTTP transport library)
Hope this helps
Oleg
> Again, thanks for your help,
> - Stephanie
>
> -----Original Message-----
> From: Oleg Kalnichevski [mailto:olegk@apache.org]
> Sent: Tuesday, April 21, 2009 6:54 AM
> To: HttpClient User Discussion
> Subject: Re: Can servers "block" programmatic form submissions?
>
> On Mon, Apr 20, 2009 at 01:03:16PM -0400, Klein, Stephanie [USA] wrote:
> > Hi, All,
> >
> > I'm writing an application to log into a website via a form. I've
> tried
> > many different ways, but each time, I get re-directed to a page saying
> > I'm being "redirected to an authentication service on another device".
> > (I'm not able to get to this page via a browser.)
> >
> > I downloaded a browser sniffer program, but when it's running, the
> > website I'm trying to log into won't load properly (via the browser).
> >
>
> I find this highly improbable. I can not think of a way for a server
> side application to tell that the IP traffic is being sniffed upon, but
> I am not a network specialist by any stretch of imagination. There must
> be something else.
>
>
> > So, is it possible for a company to disable programmatic access on the
> > server side? Can they have something set up to block this type of
> > log-in?
> >
>
> Yes, they can, and they often do. Lots of companies intentionally make
> it difficult to script the login process to their sitesi, for good
> reasons. However, there should always be a way to emulate the HTTP
> packets as emitted by the common browsers.
>
> Hope this helps.
>
> Oleg
>
>
> > Just in case it's helpful, here is the latest version of code I'm
> using.
> > Again, I've tried several versions. I'd be happy to share the other
> > versions if that will be helpful. I do have wire logging turned on
> and
> > would be happy to share those files also.
> >
> > // BEGIN OF CODE
> >
> > String response = "";
> >
> > GetMethod authget = new GetMethod("https://www.somesite.com");
> > (new HttpClient()).executeMethod(authget);
> > response = authget.getResponseBodyAsString();
> > System.out.println("Body from authget " + response);
> >
> > PostMethod authPost = new PostMethod("https://www.somesite.com");
> >
> > NameValuePair[] paramList = {
> > new NameValuePair("SMENC", "ISO-8859-1"),
> > new NameValuePair("SMLOCAE", "US-EN"),
> > new NameValuePair("TARGET", "/login/index.html"),
> > new NameValuePair("USER", "username"),
> > new NameValuePair("PASSWORD", "password"),
> > new NameValuePair("lowBandwidth", "false")
> > };
> >
> > System.out.println("Response body is " + response);
> > System.out.println("Login form post (authPost): " +
> > authPost.getStatusLine().toString());
> >
> > // END OF CODE
> >
> > Thank you for your time and consideration,
> >
> > - Stephanie
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
RE: Can servers "block" programmatic form submissions?
Posted by "Klein, Stephanie [USA]" <kl...@bah.com>.
First, thank you for taking the time to respond to my question. I
really appreciate it.
"> I downloaded a browser sniffer program, but when it's running, the
> website I'm trying to log into won't load properly (via the browser).
>
I find this highly improbable. I can not think of a way for a server
side application to tell that the IP traffic is being sniffed upon, but
I am not a network specialist by any stretch of imagination. There must
be something else."
I was rather surprised by the behavior of the website when I had the
sniffer running, too. I tried several times, though - when the sniffer
was running, the page wouldn't load. When the sniffer was stopped, the
page loaded fine. I suppose it could be something with the firewall
here at work - I'll try at home on my personal computer and see if I get
the same results.
" there should always be a way to emulate the HTTP packets as emitted by
the common browsers."
Can you suggest a particular object I should research/explore to try to
find a way to emulate the browser? Will I want to look at HttpClient,
or is there something else I should look into?
Again, thanks for your help,
- Stephanie
-----Original Message-----
From: Oleg Kalnichevski [mailto:olegk@apache.org]
Sent: Tuesday, April 21, 2009 6:54 AM
To: HttpClient User Discussion
Subject: Re: Can servers "block" programmatic form submissions?
On Mon, Apr 20, 2009 at 01:03:16PM -0400, Klein, Stephanie [USA] wrote:
> Hi, All,
>
> I'm writing an application to log into a website via a form. I've
tried
> many different ways, but each time, I get re-directed to a page saying
> I'm being "redirected to an authentication service on another device".
> (I'm not able to get to this page via a browser.)
>
> I downloaded a browser sniffer program, but when it's running, the
> website I'm trying to log into won't load properly (via the browser).
>
I find this highly improbable. I can not think of a way for a server
side application to tell that the IP traffic is being sniffed upon, but
I am not a network specialist by any stretch of imagination. There must
be something else.
> So, is it possible for a company to disable programmatic access on the
> server side? Can they have something set up to block this type of
> log-in?
>
Yes, they can, and they often do. Lots of companies intentionally make
it difficult to script the login process to their sitesi, for good
reasons. However, there should always be a way to emulate the HTTP
packets as emitted by the common browsers.
Hope this helps.
Oleg
> Just in case it's helpful, here is the latest version of code I'm
using.
> Again, I've tried several versions. I'd be happy to share the other
> versions if that will be helpful. I do have wire logging turned on
and
> would be happy to share those files also.
>
> // BEGIN OF CODE
>
> String response = "";
>
> GetMethod authget = new GetMethod("https://www.somesite.com");
> (new HttpClient()).executeMethod(authget);
> response = authget.getResponseBodyAsString();
> System.out.println("Body from authget " + response);
>
> PostMethod authPost = new PostMethod("https://www.somesite.com");
>
> NameValuePair[] paramList = {
> new NameValuePair("SMENC", "ISO-8859-1"),
> new NameValuePair("SMLOCAE", "US-EN"),
> new NameValuePair("TARGET", "/login/index.html"),
> new NameValuePair("USER", "username"),
> new NameValuePair("PASSWORD", "password"),
> new NameValuePair("lowBandwidth", "false")
> };
>
> System.out.println("Response body is " + response);
> System.out.println("Login form post (authPost): " +
> authPost.getStatusLine().toString());
>
> // END OF CODE
>
> Thank you for your time and consideration,
>
> - Stephanie
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: Can servers "block" programmatic form submissions?
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, Apr 20, 2009 at 01:03:16PM -0400, Klein, Stephanie [USA] wrote:
> Hi, All,
>
> I'm writing an application to log into a website via a form. I've tried
> many different ways, but each time, I get re-directed to a page saying
> I'm being "redirected to an authentication service on another device".
> (I'm not able to get to this page via a browser.)
>
> I downloaded a browser sniffer program, but when it's running, the
> website I'm trying to log into won't load properly (via the browser).
>
I find this highly improbable. I can not think of a way for a server side application to tell that the IP traffic is being sniffed upon, but I am not a network specialist by any stretch of imagination. There must be something else.
> So, is it possible for a company to disable programmatic access on the
> server side? Can they have something set up to block this type of
> log-in?
>
Yes, they can, and they often do. Lots of companies intentionally make it difficult to script the login process to their sitesi, for good reasons. However, there should always be a way to emulate the HTTP packets as emitted by the common browsers.
Hope this helps.
Oleg
> Just in case it's helpful, here is the latest version of code I'm using.
> Again, I've tried several versions. I'd be happy to share the other
> versions if that will be helpful. I do have wire logging turned on and
> would be happy to share those files also.
>
> // BEGIN OF CODE
>
> String response = "";
>
> GetMethod authget = new GetMethod("https://www.somesite.com");
> (new HttpClient()).executeMethod(authget);
> response = authget.getResponseBodyAsString();
> System.out.println("Body from authget " + response);
>
> PostMethod authPost = new PostMethod("https://www.somesite.com");
>
> NameValuePair[] paramList = {
> new NameValuePair("SMENC", "ISO-8859-1"),
> new NameValuePair("SMLOCAE", "US-EN"),
> new NameValuePair("TARGET", "/login/index.html"),
> new NameValuePair("USER", "username"),
> new NameValuePair("PASSWORD", "password"),
> new NameValuePair("lowBandwidth", "false")
> };
>
> System.out.println("Response body is " + response);
> System.out.println("Login form post (authPost): " +
> authPost.getStatusLine().toString());
>
> // END OF CODE
>
> Thank you for your time and consideration,
>
> - Stephanie
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org