Re: Integration with RESTful framework

[Tauren Mills <yo...@gmail.com>]

There have been a few messages recently about Shiro and Jersey that prompted
me to resurrect this old related thread.

My application currently has some wicket pages for login, logout, account
creation, and a few other things. It also uses a rememberme cookie for
authentication. Once you are logged in, there is a section of the
application that is entirely a single page javascript application. This JS
app uses ajax calls to a RESTful jersey service hosted on the same server.
The cookie works fine to authenticate for the jersey services as well.

The problem is that I'd like to move my API from http://mysite.com/api to
http://api.mysite.com/. If I do this, I won't be able to use the cookie auth
anymore. Plus, as Tamas suggested below, it is probably best to be stateless
on the server side and expect the user to auth for every REST call. Lastly,
I intend to open up the API in the future, and I'm not sure relying on
cookies is the best technique.

So, I'm just trying to wrap my head around authenticating with every REST
call. What are good methods of doing this? Does this mean you pass a
username and password as query parameters or in post data? I don't
necessarily want my JS app to have to ask for and store the user's password
(they already authenticated in the wicket app), nor do I like sending
passwords in the clear like this:
http://localhost:8080/services/helloworld?user=tauren&password=silly

I've just started looking into the ways that others such as twitter,
facebook, amazon, etc do this. So I'm reading up on OAuth and other
techniques. The Shiro issue tracker has an feature request for OAuth, but it
doesn't seem like it is a very high priority. I've also seen that many
services issue an API public/private key that an application would use,
which is an approach I might use down the road. I'd also like to support
 OpenID, Facebook Connect (or whatever they call it now), etc. At the moment
I haven't yet grokked it all, so please let me know if I'm going down the
wrong path!

Currently, my API is in total flux and hasn't been opened up. It only needs
to be usable by my own webapp for now, it doesn't need to be publicly
accessible to other applications. But in the near future, parts of it will
be opened up. I'll need to support JSONP at that point too.

My goal is to have my RESTful URLs return different information depending
upon who is accessing them.  If the request is not from an authenticated
user, then some URLs will return publicly available information only.  Some
URLs will return no data or an error if you are not logged in.  If you are
logged in, then you can request things about yourself.  And if you are
logged in and are a user with additional administrative privileges, then you
can request information about others that you have the right to see. But I
can see problems with this approach too, so it isn't set in stone yet.

Any advice or insights on how to proceed would be extremely helpful! Without
OAuth support, I'm unclear on the approaches that other are taking. It seems
like a common enough use case that there must be others doing the same
thing. Are there some good resources or open source tools I should be
looking into?

Thanks,
Tauren


2010/1/21 Tamás Cservenák <ta...@cservenak.net>

> Hi Tauren,
>
> we decided to be completely stateless on server side, and we do expect user
> auth for _every_ REST call.
>
> ~t~
>
>

Re: Integration with RESTful framework

[Les Hazlewood <lh...@apache.org>]

>> For example, in shiro.ini:
>>
>> securityManager.rememberMeManager.cookie.domain = .mysite.com
>
> Also, how is this configuration made using Spring? I'm not using INI
> config.  Is it as simple as something like this?
>
>    <bean id="securityManager"
> class="org.apache.shiro.web.mgt.DefaultWebSecurityManager">
>        <property name="rememberMeManager" ref="myRememberMeManager"/>
>    </bean>
>    <bean name="myRememberMeManager"
> class="com.mysite.security.MyRememberMeManager">
>        <property name="cookie.domain" value=".mysite.com"/>
>    </bean>

I thought Spring allows you to set nested properties via the '.'
property notation.  i.e:

<bean id="securityManager"
class="org.apache.shiro.web.mgt.DefaultWebSecurityManager">
    <property name="rememberMeManager.cookie.domain" value=".mysite.com"/>
</bean>

I *thought* this worked the last time I needed to try it, but I can't
remember now.  Of course, the above example would probably only work
if you're using the default CookieRememberMeManager.  If using a
custom RememberMeManager implementation as your example above shows,
then yes, it should work fine.

>> 1.  Authenticate once per session, and ensure the session id cookie is
>> also a subdomain cookie and then just ensure the session id
>> accompanies every request.  Granted, this does require server state
>> (Sessions), and only you will know how many sessions your server (or
>> cluster).  Just make sure you configure Shiro with a Cache if you use
>> its native session support (and a distributed cache if running more
>> than one application node in a cluster).
>
> So doing this would mean that a 3rd party app would need to direct its
> users to my login page for them to login, which would then redirect
> them back to the 3rd party app? It would have to do that each time a
> new session is created, right? The 3rd party app would need to be able
> to handle cookies, but I'm sure most platforms can already do that.

Yep that would be the way to go with the cookie approach, since the
3rd party app couldn't set cookies on behalf of your domain (browser
security restriction).  Note that even with OpenID, the 3rd-party app
would need to direct the user to your website - they would just enter
their OpenID instead of a username/password.

>> 2.  Authenticate on every request, which means Shiro will no longer
>> need to store its authentication state in the session.  If you have
>> other parts of code that rely on a Session, you'll still need to go
>> with an approach like #1.
>
> This is where I get confused on how to proceed. If my API requires
> auth on every request, wouldn't the 3rd party app need to somehow
> store that auth information and send it with every request? What are
> some good methods of doing this in a secure manner?
>
> Let me be clear here... when I say 3rd party app, I mean several
> different things:
>
> 1. A mobile client (iPhone, Android, Blackberry, Symbian, etc.)
> 2. A web-based JAVASCRIPT client, without a server-side component,
> hosted on someone else's server.
> 3. A RIA client, such as an AIR app or something similar
>
> If the application is just a single page of Javascript, it doesn't
> seem safe for it to request a username and password and store them in
> JS variables to send with each request. I'm starting to think that a
> server-side component might be a requirement to do this right for
> 3rd-party apps.

Yes, there is some concern about that, given that you can't control
the client tier.  If you enforce TLS connections, it becomes a little
more secure since the traffic is encrypted, but you can't guarantee
that the client itself is secure.  In other words, TLS guarantess a
point-to-point (host to host) secure channel, but it doesn't guarantee
end-to-end (app-to-app) security.

WS-Security however was created to help address this gap.  I've heard
of people using WS-Security with REST by including the WS-Security
information as an HTTP header.  That is, you don't have to use SOAP to
use WS-Security, but you would need some custom coding to handle this
special case for your REST calls.

So, if you want things to be as secure as possible, the client has to
understand these mechanisms - either by performing strong encryption
itself, or by using a WS-Security library or something similar.  Or it
can simulate what a web browser does with HTTP Digest authentication.
Or you use TLS Mutual Authentication (aka client ssl certificates).

Including username/password pairs in HTTP headers and sending them
over a non-mutual (server certificate only) TLS connection might work,
but it is not as secure as the above options.  But these options place
mandates on your client tier - you will have to decide if you can
mandate these options, given the clients you have to support.

> I guess there are two approaches:
>
> 1. The 3rd party app runs completely in the client, and the client
> makes requests directly to my API

Yep, but see the above comments about the client communicating securely.

>
> 2. The 3rd party app runs in the client, but makes requests to a
> 3rd-party server component. The server component makes requests to my
> API.

I think this is where TLS mutual authentication is a best practice
(IMO) - both servers would trust each other, and can ensure that all
data is transmitted safely.  But there is an additional network hop
(client -> 3rd party server -> your server) instead of just client ->
your server.  The extra hop may or may not be acceptable depending on
your performance requirements.

And you probably wouldn't want any of your system's username/password
pairs being sent by the client to the 3rd party server since you
couldn't guarantee that they would remain safe (you can't control how
the 3rd party client and the 3rd party server function together).

> So I assume your 3rd party applications are running on a server, and
> that your secret key is protected from the public?

No, they can be desktop apps, but they are controlled - the apps do
not openly share their id/key with end-users (i.e. it is part of the
app's codebase and/or private config).

Sorry if my reply only muddied the waters, but hopefully it will give
you some ideas!

Best,

Les

Re: Integration with RESTful framework

[Tauren Mills <ta...@tauren.com>]

Les,

Thanks for your insights. See inline below.

> Just a quick note - rememberMe cookies do not perform authentication -
> they just remember an identity, and that identity is _assumed_ to be
> correct on subsequent requests.  Until someone logs-in, they haven't
> proved their identity, so we can't call this legitimate
> 'authentication'.  I know you probably know this already since you've
> used Shiro's API for a while now, I just want to point this out for
> anyone who comes across this thread in the future.

Yes, I'm fully aware of this, but thanks for pointing it out.

> You could probably use a subdomain cookie that can be shared across
> hosts in the same domain.  Just set the 'domain' cookie property to
> .mysite.com  (notice the leading '.').  For example, in shiro.ini:
>
> securityManager.rememberMeManager.cookie.domain = .mysite.com

This solution would certainly be easiest to meet my current needs, and
I will probably do this initially for my single-page webapp version.
But would it help for a public API that has 3rd party applications
connecting to it?

Also, how is this configuration made using Spring? I'm not using INI
config.  Is it as simple as something like this?

    <bean id="securityManager"
class="org.apache.shiro.web.mgt.DefaultWebSecurityManager">
        <property name="rememberMeManager" ref="myRememberMeManager"/>
    </bean>
    <bean name="myRememberMeManager"
class="com.mysite.security.MyRememberMeManager">
    	<property name="cookie.domain" value=".mysite.com"/>
    </bean>

> I'm not sure this is a bad idea, but if you didn't want to go this
> route, you have a few options:
>
> 1.  Authenticate once per session, and ensure the session id cookie is
> also a subdomain cookie and then just ensure the session id
> accompanies every request.  Granted, this does require server state
> (Sessions), and only you will know how many sessions your server (or
> cluster).  Just make sure you configure Shiro with a Cache if you use
> its native session support (and a distributed cache if running more
> than one application node in a cluster).

So doing this would mean that a 3rd party app would need to direct its
users to my login page for them to login, which would then redirect
them back to the 3rd party app? It would have to do that each time a
new session is created, right? The 3rd party app would need to be able
to handle cookies, but I'm sure most platforms can already do that.

> 2.  Authenticate on every request, which means Shiro will no longer
> need to store its authentication state in the session.  If you have
> other parts of code that rely on a Session, you'll still need to go
> with an approach like #1.

This is where I get confused on how to proceed. If my API requires
auth on every request, wouldn't the 3rd party app need to somehow
store that auth information and send it with every request? What are
some good methods of doing this in a secure manner?

Let me be clear here... when I say 3rd party app, I mean several
different things:

1. A mobile client (iPhone, Android, Blackberry, Symbian, etc.)
2. A web-based JAVASCRIPT client, without a server-side component,
hosted on someone else's server.
3. A RIA client, such as an AIR app or something similar

If the application is just a single page of Javascript, it doesn't
seem safe for it to request a username and password and store them in
JS variables to send with each request. I'm starting to think that a
server-side component might be a requirement to do this right for
3rd-party apps.

I guess there are two approaches:

1. The 3rd party app runs completely in the client, and the client
makes requests directly to my API
2. The 3rd party app runs in the client, but makes requests to a
3rd-party server component. The server component makes requests to my
API.

I'm just trying to determine what the best practices are for these
different types of scenarios. My goal is to simply expose a RESTful
API, and all clients would use it, including my own web-based and
mobile clients.

> My company's product currently uses #2 as the approach with REST based
> services - it is an Amazon EC2-like model:
>
> Each API Consumer (essentially a 3rd-party application) that is
> allowed to call into my system is assigned its own 'Access ID' (think
> of this as a consumer-specific 'userername') and a corresponding
> 'Secret Key' (essentially an application-level 'password').  When an
> API consumer is registered within my application, I auto generate its
> Access ID and Secret Key - they're both just randomly generated UUIDs
> with the dashes stripped out.  The Access ID is shown in the UI freely
> - the Secret Key is hidden unless the end-user requests to see it.
>
> Each request that comes in to the system specifies both the Access ID
> and Secret Key as HTTP headers (all of this is encrypted over a TLS
> connection).  I then have a servlet filter that extracts them,
> performs the authentication attempt, and if successful, allows the
> request to continue through the filter chain.  If it fails, I
> immediately return an HTTP 401 Unauthorized status code.

So I assume your 3rd party applications are running on a server, and
that your secret key is protected from the public?

> 3.  However, I'm re-thinking this - HTTP Digest authentication
> essentially is the same thing in a more secure way and HTTPS isn't
> required.  That might be a better approach than #2.
>
> 4.  Potentially better than any of these approaches would be TLS
> mutual authentication, but this has been discussed in a few forums,
> saying that it incurs a heavy performance hit due to the mutual
> handshake.  It is pretty rock solid though, so pick your poison.

I'll look into this more.

>> Any advice or insights on how to proceed would be extremely helpful! Without
>> OAuth support, I'm unclear on the approaches that other are taking.
>
> I can't speak for all of the dev team - it would have to be discussed
> - but I wouldn't have any objections to supporting OAuth in Shiro if
> the community wanted it.  It is just low priority for me personally,
> so that's why I don't work on it.  If anyone would like to help us
> work on this, I'm sure we'd be quite open to suggestions on the dev
> list :)

I've barely got me feet wet yet in researching OAuth, but from the
little I've seen, it appears it would help solve some problems. I'm
going to research this further and perhaps offer some suggestions.

Thanks again for your thoughts.

Tauren

Re: Integration with RESTful framework

[Les Hazlewood <lh...@apache.org>]

Hi Tauren,

Please see inline.

> There have been a few messages recently about Shiro and Jersey that prompted
> me to resurrect this old related thread.
> My application currently has some wicket pages for login, logout, account
> creation, and a few other things. It also uses a rememberme cookie for
> authentication.

Just a quick note - rememberMe cookies do not perform authentication -
they just remember an identity, and that identity is _assumed_ to be
correct on subsequent requests.  Until someone logs-in, they haven't
proved their identity, so we can't call this legitimate
'authentication'.  I know you probably know this already since you've
used Shiro's API for a while now, I just want to point this out for
anyone who comes across this thread in the future.

> Once you are logged in, there is a section of the
> application that is entirely a single page javascript application. This JS
> app uses ajax calls to a RESTful jersey service hosted on the same server.
> The cookie works fine to authenticate for the jersey services as well.
> The problem is that I'd like to move my API from http://mysite.com/api to
> http://api.mysite.com/. If I do this, I won't be able to use the cookie auth
> anymore. Plus, as Tamas suggested below, it is probably best to be stateless
> on the server side and expect the user to auth for every REST call. Lastly,
> I intend to open up the API in the future, and I'm not sure relying on
> cookies is the best technique.

You could probably use a subdomain cookie that can be shared across
hosts in the same domain.  Just set the 'domain' cookie property to
.mysite.com  (notice the leading '.').  For example, in shiro.ini:

securityManager.rememberMeManager.cookie.domain = .mysite.com

I'm not sure this is a bad idea, but if you didn't want to go this
route, you have a few options:

1.  Authenticate once per session, and ensure the session id cookie is
also a subdomain cookie and then just ensure the session id
accompanies every request.  Granted, this does require server state
(Sessions), and only you will know how many sessions your server (or
cluster).  Just make sure you configure Shiro with a Cache if you use
its native session support (and a distributed cache if running more
than one application node in a cluster).

2.  Authenticate on every request, which means Shiro will no longer
need to store its authentication state in the session.  If you have
other parts of code that rely on a Session, you'll still need to go
with an approach like #1.

My company's product currently uses #2 as the approach with REST based
services - it is an Amazon EC2-like model:

Each API Consumer (essentially a 3rd-party application) that is
allowed to call into my system is assigned its own 'Access ID' (think
of this as a consumer-specific 'userername') and a corresponding
'Secret Key' (essentially an application-level 'password').  When an
API consumer is registered within my application, I auto generate its
Access ID and Secret Key - they're both just randomly generated UUIDs
with the dashes stripped out.  The Access ID is shown in the UI freely
- the Secret Key is hidden unless the end-user requests to see it.

Each request that comes in to the system specifies both the Access ID
and Secret Key as HTTP headers (all of this is encrypted over a TLS
connection).  I then have a servlet filter that extracts them,
performs the authentication attempt, and if successful, allows the
request to continue through the filter chain.  If it fails, I
immediately return an HTTP 401 Unauthorized status code.

3.  However, I'm re-thinking this - HTTP Digest authentication
essentially is the same thing in a more secure way and HTTPS isn't
required.  That might be a better approach than #2.

4.  Potentially better than any of these approaches would be TLS
mutual authentication, but this has been discussed in a few forums,
saying that it incurs a heavy performance hit due to the mutual
handshake.  It is pretty rock solid though, so pick your poison.

> Any advice or insights on how to proceed would be extremely helpful! Without
> OAuth support, I'm unclear on the approaches that other are taking.

I can't speak for all of the dev team - it would have to be discussed
- but I wouldn't have any objections to supporting OAuth in Shiro if
the community wanted it.  It is just low priority for me personally,
so that's why I don't work on it.  If anyone would like to help us
work on this, I'm sure we'd be quite open to suggestions on the dev
list :)

Cheers,

Les