You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Pithon Philippe <pp...@gmail.com> on 2016/06/07 06:41:10 UTC
Solr 5.4 Transaction
Hi,
I have a question on Solr Transaction as relational databases
The Solr commit is not isolated for each client session, right?
In my test (source below) The commit in a session adds records of other
sessions
is there a documentation on this?
is what's planned improvements on this? version 6, version 7?
Thank you for any ideas!
Source example :
public class TransactionTest {
static final String BASE_URL = "http://localhost:8983/solr/test";
public static void main(String[] args) {
try {
new TransactionTest();
} catch (Exception e) {
e.printStackTrace();
}
}
public TransactionTest() throws Exception {
HttpSolrClient solrClient = new HttpSolrClient(BASE_URL);
DTOMail mail = new DTOMail();
mail.setType("mail");
mail.setBody("test body");
System.out.println("add been");
solrClient.addBean(mail);
pause();
System.out.println("commit");
solrClient.commit();
solrClient.close();
}
private void pause() {
try {
System.in.read();
} catch (Exception e) {
}
}
}
Re: Solr 5.4 Transaction
Posted by Shawn Heisey <ap...@elyograg.org>.
On 6/7/2016 12:41 AM, Pithon Philippe wrote:
> I have a question on Solr Transaction as relational databases The Solr
> commit is not isolated for each client session, right? In my test
> (source below) The commit in a session adds records of other sessions
> is there a documentation on this? is what's planned improvements on
> this? version 6, version 7?
No improvements are planned.
You've gotten a few replies telling you that if you want real
transactions in Solr, you'll have to implement them. This is because
Lucene (the API that provides most of Solr's functionality) does not do
transactions like a relational database does. That's the important
thing to remember here. Lucene is a *search* API, not a *database*
API. Solr is a search server, not a database server.
Adding the kind of transactions you want would require fundamental (and
likely large-scale) changes to the low-level design, quite possibly at
the Lucene level rather than Solr. There's a very good chance that
adding this would slow down the primary feature -- search. It would
also require considerable effort -- both to initially write and to
maintain/debug.
Solr is advertised as NoSQL. If they are configured and used correctly,
Lucene-based applications (like Solr) can fill a NoSQL role, but this is
not the GOAL of the software. I get a little worried every time I see
the release note claiming NoSQL capability, because when they see that,
I know that users are going to request additional features that search
doesn't need, like transactions.
Something else that I saw in the replies you already got: If you really
need transaction support, find existing software that's been doing it
for a lot of years, and use that software as a canonical data source to
populate your Solr index.
Thanks,
Shawn
Re: Solr 5.4 Transaction
Posted by Upayavira <uv...@odoko.co.uk>.
There is no equivalent to transactions in Lucene or Solr. A commit
persists newly added documents to disk - all of them, regardless of
which client sent them. A rollback discards all uncommitted documents,
regardless of which client sent them, which renders a rollback pretty
un-useful.
If you need a transaction system, you're gonna need to manage it
yourself. You could, before updating some documents, retrieve and store
them. Then, should the transaction fail, you post your old documents
over the top of your changes. You might be able to use some of the
features of partial updates (e.g. the _version_ field) to ensure that
you don't accidentally overwrite someone else's changes.
But basically, if you want a transaction system, you're gonna have to
build it yourself. (Or, push the data to a database that *does* support
transactions, and then have a process that pulls updates from the DB to
Solr).
Upayavira
On Tue, 7 Jun 2016, at 10:52 AM, Vincenzo D'Amore wrote:
> Hi Pithon,
>
> I have to state beforehand that I worked with transactions on Solr 4.8.1,
> so I'm not sure the transactions are changed in Solr over time.
> And, I have to add, the transactions support not is very well documented,
> so most of what I know is based on my experience.
>
> Given that, Solr clients connects via HTTP, which is a stateless
> protocol,
> so many clients can connect concurrently.
> That's important point, and IMHO it make a great difference from other
> products that have clients with an "open connection", which enable them
> to
> have a transaction at protocol level.
>
> So this means that when a client commit (or rollback) to all active
> clients are involved.
> When a client rollbacks, all the previous work done by all the clients
> will
> be rollbacked.
>
> That's why, if you really need to rollback, only one client should be
> connected at a time.
>
> I think, if you really want implement transactions with Solr, what you
> can
> do is add a layer, your own layer, that transparently give this
> functionality.
>
> Best regards,
> Vincenzo
>
>
>
> On Tue, Jun 7, 2016 at 11:14 AM, Pithon Philippe
> <pp...@gmail.com>
> wrote:
>
> > Thanks,
> > my problem is for rollback transaction by user...
> > if there is a problem in trycatch for a user, the rollback run for all
> > users commit...
> > Philippe
> >
> > 2016-06-07 9:23 GMT+02:00 Mikhail Khludnev <mk...@griddynamics.com>:
> >
> > > Hello,
> > >
> > > That's how Lucene work underneath.
> > > Just ad-hoc idea you can have isolated per session indices and then add
> > > them altogether. Thinking deeper, we can consider per thread invertors
> > > which somehow isolated, perhaps something transactional might be built on
> > > top of them, but it's really deep hack.
> > > Anyway, it.s not clear what you want to achieve and why by this way.
> > Also,
> > > this question suits for dev@ much more.
> > > 07 июня 2016 г. 9:41 пользователь "Pithon Philippe" <
> > > ppithon.sigal@gmail.com>
> > > написал:
> > >
> > > > Hi,
> > > > I have a question on Solr Transaction as relational databases
> > > >
> > > > The Solr commit is not isolated for each client session, right?
> > > > In my test (source below) The commit in a session adds records of other
> > > > sessions
> > > > is there a documentation on this?
> > > > is what's planned improvements on this? version 6, version 7?
> > > > Thank you for any ideas!
> > > >
> > > >
> > > > Source example :
> > > >
> > > > public class TransactionTest {
> > > >
> > > > static final String BASE_URL = "http://localhost:8983/solr/test";
> > > >
> > > > public static void main(String[] args) {
> > > >
> > > > try {
> > > > new TransactionTest();
> > > > } catch (Exception e) {
> > > > e.printStackTrace();
> > > > }
> > > >
> > > > }
> > > >
> > > > public TransactionTest() throws Exception {
> > > >
> > > > HttpSolrClient solrClient = new HttpSolrClient(BASE_URL);
> > > >
> > > > DTOMail mail = new DTOMail();
> > > > mail.setType("mail");
> > > > mail.setBody("test body");
> > > >
> > > > System.out.println("add been");
> > > > solrClient.addBean(mail);
> > > >
> > > > pause();
> > > >
> > > > System.out.println("commit");
> > > > solrClient.commit();
> > > >
> > > > solrClient.close();
> > > > }
> > > >
> > > > private void pause() {
> > > >
> > > > try {
> > > > System.in.read();
> > > > } catch (Exception e) {
> > > > }
> > > >
> > > > }
> > > >
> > > > }
> > > >
> > >
> >
>
>
>
> --
> Vincenzo D'Amore
> email: v.damore@gmail.com
> skype: free.dev
> mobile: +39 349 8513251
Re: Solr 5.4 Transaction
Posted by Vincenzo D'Amore <v....@gmail.com>.
Hi Pithon,
I have to state beforehand that I worked with transactions on Solr 4.8.1,
so I'm not sure the transactions are changed in Solr over time.
And, I have to add, the transactions support not is very well documented,
so most of what I know is based on my experience.
Given that, Solr clients connects via HTTP, which is a stateless protocol,
so many clients can connect concurrently.
That's important point, and IMHO it make a great difference from other
products that have clients with an "open connection", which enable them to
have a transaction at protocol level.
So this means that when a client commit (or rollback) to all active
clients are involved.
When a client rollbacks, all the previous work done by all the clients will
be rollbacked.
That's why, if you really need to rollback, only one client should be
connected at a time.
I think, if you really want implement transactions with Solr, what you can
do is add a layer, your own layer, that transparently give this
functionality.
Best regards,
Vincenzo
On Tue, Jun 7, 2016 at 11:14 AM, Pithon Philippe <pp...@gmail.com>
wrote:
> Thanks,
> my problem is for rollback transaction by user...
> if there is a problem in trycatch for a user, the rollback run for all
> users commit...
> Philippe
>
> 2016-06-07 9:23 GMT+02:00 Mikhail Khludnev <mk...@griddynamics.com>:
>
> > Hello,
> >
> > That's how Lucene work underneath.
> > Just ad-hoc idea you can have isolated per session indices and then add
> > them altogether. Thinking deeper, we can consider per thread invertors
> > which somehow isolated, perhaps something transactional might be built on
> > top of them, but it's really deep hack.
> > Anyway, it.s not clear what you want to achieve and why by this way.
> Also,
> > this question suits for dev@ much more.
> > 07 июня 2016 г. 9:41 пользователь "Pithon Philippe" <
> > ppithon.sigal@gmail.com>
> > написал:
> >
> > > Hi,
> > > I have a question on Solr Transaction as relational databases
> > >
> > > The Solr commit is not isolated for each client session, right?
> > > In my test (source below) The commit in a session adds records of other
> > > sessions
> > > is there a documentation on this?
> > > is what's planned improvements on this? version 6, version 7?
> > > Thank you for any ideas!
> > >
> > >
> > > Source example :
> > >
> > > public class TransactionTest {
> > >
> > > static final String BASE_URL = "http://localhost:8983/solr/test";
> > >
> > > public static void main(String[] args) {
> > >
> > > try {
> > > new TransactionTest();
> > > } catch (Exception e) {
> > > e.printStackTrace();
> > > }
> > >
> > > }
> > >
> > > public TransactionTest() throws Exception {
> > >
> > > HttpSolrClient solrClient = new HttpSolrClient(BASE_URL);
> > >
> > > DTOMail mail = new DTOMail();
> > > mail.setType("mail");
> > > mail.setBody("test body");
> > >
> > > System.out.println("add been");
> > > solrClient.addBean(mail);
> > >
> > > pause();
> > >
> > > System.out.println("commit");
> > > solrClient.commit();
> > >
> > > solrClient.close();
> > > }
> > >
> > > private void pause() {
> > >
> > > try {
> > > System.in.read();
> > > } catch (Exception e) {
> > > }
> > >
> > > }
> > >
> > > }
> > >
> >
>
--
Vincenzo D'Amore
email: v.damore@gmail.com
skype: free.dev
mobile: +39 349 8513251
Re: Solr 5.4 Transaction
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
Ah, it's easy. It reminds me the fact that ESB don't use transactions in
favor of compansations.
Allocate some txId, assign it on every doc then if you need to rollback
submit deleteByQuery txId:<666>, and commit. Although, it provides phantom
and dirty reads both, it's really cheap. If you need something super atomic
evaluate addIndex approach.
On Tue, Jun 7, 2016 at 12:14 PM, Pithon Philippe <pp...@gmail.com>
wrote:
> Thanks,
> my problem is for rollback transaction by user...
> if there is a problem in trycatch for a user, the rollback run for all
> users commit...
> Philippe
>
> 2016-06-07 9:23 GMT+02:00 Mikhail Khludnev <mk...@griddynamics.com>:
>
> > Hello,
> >
> > That's how Lucene work underneath.
> > Just ad-hoc idea you can have isolated per session indices and then add
> > them altogether. Thinking deeper, we can consider per thread invertors
> > which somehow isolated, perhaps something transactional might be built on
> > top of them, but it's really deep hack.
> > Anyway, it.s not clear what you want to achieve and why by this way.
> Also,
> > this question suits for dev@ much more.
> > 07 июня 2016 г. 9:41 пользователь "Pithon Philippe" <
> > ppithon.sigal@gmail.com>
> > написал:
> >
> > > Hi,
> > > I have a question on Solr Transaction as relational databases
> > >
> > > The Solr commit is not isolated for each client session, right?
> > > In my test (source below) The commit in a session adds records of other
> > > sessions
> > > is there a documentation on this?
> > > is what's planned improvements on this? version 6, version 7?
> > > Thank you for any ideas!
> > >
> > >
> > > Source example :
> > >
> > > public class TransactionTest {
> > >
> > > static final String BASE_URL = "http://localhost:8983/solr/test";
> > >
> > > public static void main(String[] args) {
> > >
> > > try {
> > > new TransactionTest();
> > > } catch (Exception e) {
> > > e.printStackTrace();
> > > }
> > >
> > > }
> > >
> > > public TransactionTest() throws Exception {
> > >
> > > HttpSolrClient solrClient = new HttpSolrClient(BASE_URL);
> > >
> > > DTOMail mail = new DTOMail();
> > > mail.setType("mail");
> > > mail.setBody("test body");
> > >
> > > System.out.println("add been");
> > > solrClient.addBean(mail);
> > >
> > > pause();
> > >
> > > System.out.println("commit");
> > > solrClient.commit();
> > >
> > > solrClient.close();
> > > }
> > >
> > > private void pause() {
> > >
> > > try {
> > > System.in.read();
> > > } catch (Exception e) {
> > > }
> > >
> > > }
> > >
> > > }
> > >
> >
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
<http://www.griddynamics.com>
<mk...@griddynamics.com>
Re: Solr 5.4 Transaction
Posted by Pithon Philippe <pp...@gmail.com>.
Thanks,
my problem is for rollback transaction by user...
if there is a problem in trycatch for a user, the rollback run for all
users commit...
Philippe
2016-06-07 9:23 GMT+02:00 Mikhail Khludnev <mk...@griddynamics.com>:
> Hello,
>
> That's how Lucene work underneath.
> Just ad-hoc idea you can have isolated per session indices and then add
> them altogether. Thinking deeper, we can consider per thread invertors
> which somehow isolated, perhaps something transactional might be built on
> top of them, but it's really deep hack.
> Anyway, it.s not clear what you want to achieve and why by this way. Also,
> this question suits for dev@ much more.
> 07 июня 2016 г. 9:41 пользователь "Pithon Philippe" <
> ppithon.sigal@gmail.com>
> написал:
>
> > Hi,
> > I have a question on Solr Transaction as relational databases
> >
> > The Solr commit is not isolated for each client session, right?
> > In my test (source below) The commit in a session adds records of other
> > sessions
> > is there a documentation on this?
> > is what's planned improvements on this? version 6, version 7?
> > Thank you for any ideas!
> >
> >
> > Source example :
> >
> > public class TransactionTest {
> >
> > static final String BASE_URL = "http://localhost:8983/solr/test";
> >
> > public static void main(String[] args) {
> >
> > try {
> > new TransactionTest();
> > } catch (Exception e) {
> > e.printStackTrace();
> > }
> >
> > }
> >
> > public TransactionTest() throws Exception {
> >
> > HttpSolrClient solrClient = new HttpSolrClient(BASE_URL);
> >
> > DTOMail mail = new DTOMail();
> > mail.setType("mail");
> > mail.setBody("test body");
> >
> > System.out.println("add been");
> > solrClient.addBean(mail);
> >
> > pause();
> >
> > System.out.println("commit");
> > solrClient.commit();
> >
> > solrClient.close();
> > }
> >
> > private void pause() {
> >
> > try {
> > System.in.read();
> > } catch (Exception e) {
> > }
> >
> > }
> >
> > }
> >
>
Re: Solr 5.4 Transaction
Posted by Mikhail Khludnev <mk...@griddynamics.com>.
Hello,
That's how Lucene work underneath.
Just ad-hoc idea you can have isolated per session indices and then add
them altogether. Thinking deeper, we can consider per thread invertors
which somehow isolated, perhaps something transactional might be built on
top of them, but it's really deep hack.
Anyway, it.s not clear what you want to achieve and why by this way. Also,
this question suits for dev@ much more.
07 июня 2016 г. 9:41 пользователь "Pithon Philippe" <pp...@gmail.com>
написал:
> Hi,
> I have a question on Solr Transaction as relational databases
>
> The Solr commit is not isolated for each client session, right?
> In my test (source below) The commit in a session adds records of other
> sessions
> is there a documentation on this?
> is what's planned improvements on this? version 6, version 7?
> Thank you for any ideas!
>
>
> Source example :
>
> public class TransactionTest {
>
> static final String BASE_URL = "http://localhost:8983/solr/test";
>
> public static void main(String[] args) {
>
> try {
> new TransactionTest();
> } catch (Exception e) {
> e.printStackTrace();
> }
>
> }
>
> public TransactionTest() throws Exception {
>
> HttpSolrClient solrClient = new HttpSolrClient(BASE_URL);
>
> DTOMail mail = new DTOMail();
> mail.setType("mail");
> mail.setBody("test body");
>
> System.out.println("add been");
> solrClient.addBean(mail);
>
> pause();
>
> System.out.println("commit");
> solrClient.commit();
>
> solrClient.close();
> }
>
> private void pause() {
>
> try {
> System.in.read();
> } catch (Exception e) {
> }
>
> }
>
> }
>