Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

One of the important things for my use of my changes to the ServiceRegistrar interface was separating deserialization of the Entrys from the service object.  Ultimately, I wanted to not have to worry about how the "codebase" was structured in order to minimize downloading. Some Jini deployers have done things to put a small jar at the front of the codebase that just has the preferred.list in it.  Even that download, for me, would have been too much.  My customers' deployments have nearly 50 codebases visible when my client starts up.  Those 50 connections over a cellular or other high latency network would make service selection take way too long.

Being able to defer downloading was also controlled by the changes to ClassLoading to include the "neverPrefer" settings on a class name.  I can do that now through the recent RMIClassLoaderSPI override capabilities.  But I still need to be able to control which Entry is deserialized.  I need to be able to ask what classes, by name, are in the namespace of each Entry object as my implementation allowed. I had made changes to Reggie marshalling to use reflection to get the class hierarchy, and then provided access to the list of classnames as part of the returned, marshalled value in my API.

Gregg Wonderly

On Jan 30, 2011, at 2:23 PM, Peter Firmstone wrote:

> Gut feel, tells me the following class is not right, it's intent is to provide an api for implementation of delayed unmarshalling and provisioning of codebases, for services implementing a lookup service or ServiceRegistrar, whilst retaining backward compatibility.
> 
> In other words, the lookup service proxy would implement it.
> 
> Thoughts I had were to declare IOException's on methods, and to return a String annotation, rather than a URI.
> 
> What are your thoughts?
> 
> Cheers,
> 
> Peter.
> 
> /*
> * Licensed to the Apache Software Foundation (ASF) under one
> * or more contributor license agreements.  See the NOTICE file
> * distributed with this work for additional information
> * regarding copyright ownership. The ASF licenses this file
> * to you under the Apache License, Version 2.0 (the
> * "License"); you may not use this file except in compliance
> * with the License. You may obtain a copy of the License at
> *
> *      http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> 
> package org.apache.river.api.lookup;
> 
> import java.net.URI;
> import java.security.CodeSource;
> import net.jini.core.entry.Entry;
> import net.jini.core.lookup.ServiceID;
> import net.jini.core.lookup.ServiceItem;
> 
> /**
> * MarshalledServiceItem extends ServiceItem and can be used anywhere a
> * ServiceItem can.  A MarshalledServiceItem implementation instance
> * contains the marshalled form of a Service and it's Entry's,
> * the corresponding superclass ServiceItem however contains null values
> * for the service and can exclude any Entry's, however where Entry
> * classes already exist at the client, that they be unmarshalled.
> *
> * The ServiceID shall be in unmarshalled form always in the ServiceItem super class.
> *
> * Since the ServiceItem.service is null, use of this class in existing software
> * will not return the service, however it will not break that software as
> * ServiceItem's contract is to set service or Entry's to null when they cannot
> * be unmarshalled.
> *
> * ServiceItem's toString() method will return a different result for
> * MarshalledServiceItem instances.
> *
> * If required, a new ServiceItem that is fully unmarshalled
> * can be constructed from this class's methods and ServiceID.
> *
> * @author Peter Firmstone.
> */
> public abstract class MarshalledServiceItem extends ServiceItem{
>   private static final long SerialVersionUID = 1L;
>   protected MarshalledServiceItem(ServiceID id, Entry[] unmarshalledEntries){
>       super(id, (Object) null, unmarshalledEntries);
>   }
>   /**
>    * Unmarshall the service proxy.
>    * @param load service with local or existing CodeSource or null for
>    * default.
>    * @return the service proxy, null if class not found.
>    */
>   public abstract Object getService(CodeSource[] code);
>   /**
>    * Unmarshall the Entry's
>    * @return array of Entry's, null entry in array for any class not found.
>    */
>   public abstract Entry[] getEntries();
>     public abstract URI[] getAnnotations();
> }
> 

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Utility class for ServiceItemFilter's discussed in previous post.

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.river.api.lookup;

import java.io.IOException;
import org.apache.river.api.util.ResultStream;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import net.jini.core.lookup.ServiceItem;
import net.jini.lookup.ServiceItemFilter;

/**
 * A utility class designed for building complex queries, from existing 
filters.
 *
 *  Filters can be daisy chained with pre prepared filters to perform 
logical AND operations.
 *
 * Logical OR operations can be simulated by providing multiple filters in
 * constructors.
 *
 * Any references to ServiceResultStreamFilter should be set to null
 * immediately after filtering.
 * New instances can be created as required.
 *
 * @author Peter Firmstone.
 */
public class ServiceResultStreamFilter implements 
ResultStream<ServiceItem> {
    private final List<ServiceItemFilter> filters;
    private final ResultStream inputResultStream;
   
    /**
     * Note the methods of ServiceResultStreamFilter implement
     * ResultStream<ServiceItem>, but the constructor doesn't, this is to
     * protect the client against unchecked type casts that would occur
     * if a ResultStream<ServiceItem> was obtained from a service.
     *
     * All methods in this implementation perform their own type safety
     * checks in order to implement ResultStream<ServiceItem> safely.
     *
     * @param rs
     * @param sf
     */
    public ServiceResultStreamFilter(ResultStream rs,
            ServiceItemFilter[] sf){
        inputResultStream = rs;
        filters = new ArrayList<ServiceItemFilter>(sf.length);
        filters.addAll(Arrays.asList(sf));
    }

    public ServiceItem get() throws IOException {
        for(Object item = inputResultStream.get(); item != null;
                item = inputResultStream.get()) {
            if (item instanceof ServiceItem){
                ServiceItem it = (ServiceItem) item;
                int l = filters.size();
                for ( int i = 0; i < l; i++){
                    ServiceItemFilter filter = filters.get(i);
                    if (filter == null) continue;
                    if (filter.check(it))  return it;
                }// end filter loop
            }// If it isn't a ServiceItem it is ignored.
        }//end item loop
        return null; // Our stream terminated item was null;
    }

    public void close() throws IOException {
        inputResultStream.close();
    }
}

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Gregg Wonderly wrote:
> One of the important things for my use of my changes to the ServiceRegistrar interface was separating deserialization of the Entrys from the service object.  Ultimately, I wanted to not have to worry about how the "codebase" was structured in order to minimize downloading. Some Jini deployers have done things to put a small jar at the front of the codebase that just has the preferred.list in it.  Even that download, for me, would have been too much.  My customers' deployments have nearly 50 codebases visible when my client starts up.  Those 50 connections over a cellular or other high latency network would make service selection take way too long.
>
> Being able to defer downloading was also controlled by the changes to ClassLoading to include the "neverPrefer" settings on a class name.  I can do that now through the recent RMIClassLoaderSPI override capabilities.  But I still need to be able to control which Entry is deserialized.  I need to be able to ask what classes, by name, are in the namespace of each Entry object as my implementation allowed. I had made changes to Reggie marshalling to use reflection to get the class hierarchy, and then provided access to the list of classnames as part of the returned, marshalled value in my API.
>
> Gregg Wonderly
>
> On Jan 30, 2011, at 2:23 PM, Peter Firmstone wrote:
>   
>
Thanks Gregg,

What follows, is an interface for a lookup service, that extends the 
existing ServiceRegistrar lookup service interface, enabling existing 
clients to utilise the existing discovery mechanisms and interfaces.

I researched your implementation before writing this, so I was strongly 
influenced by your lookup service, delayed umarshalling and selective 
unmarshalling of Entry's are very important features, missing from River.

I wanted to utilise the features you created, at the same time, I wanted 
a lookup service for the internet, so it also needed to be able to 
retrieve an almost infinite collection of results incrementally, over 
time, that may contain duplicates using a ResultStream (somewhat similar 
to MatchSet in javaspaces).  In the lookup method below, the caller can 
provide the Class files of any Entry's it's interested in, after first 
retrieving and determining them using a ServiceTemplate and 
ResultStream.  A ResultStream can use filter chaining where each filter 
performs a single simple operation (like proxy verification, 
unmarshalling, Entry based comparison, or removal), and can utilise the 
existing ServiceItemFilter interface (when applicable), uninteresting 
services can be removed from the ResultStream, or categorised and placed 
in different queues for parallel processing, or left in the stream for 
serial processing.

Once you've built a filter based result stream, this can be reused over 
and over, to process lookup results from multiple lookup services.

I have plans to create some DNS-SRV discovery support classes, used to 
produce LookupLocator's (actually a subclass of LookupLocator), to be 
used by LookupLocatorDiscovery, using Unicast discovery to find lookup 
services all over the internet.  Each domain should be responsible for 
providing lookup services for all public Jini Services it provides.

Another piece in the puzzle relates to ensuring we're reasonably 
protected against unmarshalling DOS attacks, before proxy trust is 
established.

So what I've got here is a small part of a much larger puzzle, I'm very 
interested to hear your thoughts, have your input and make changes based 
on your recommendations, and admit that the interface below has been 
inspired by your work that long preceded it.

Best regards & many thanks,

Peter.

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.river.api.lookup;

import java.io.IOException;
import net.jini.core.lookup.ServiceRegistrar;
import net.jini.core.lookup.ServiceTemplate;
import net.jini.core.lookup.ServiceItem;
import org.apache.river.api.util.ResultStream;

/**
 * Defines the interface to the lookup service.  The interface is not a
 * remote interface; each implementation of the lookup service exports
 * proxy objects that implement the StreamServiceRegistrar interface 
local to
 * the client, using an implementation-specific protocol to communicate
 * with the actual remote server.  All of the proxy methods obey normal
 * RMI remote interface semantics except where explicitly noted.  Two
 * proxy objects are equal if they are proxies for the same lookup service.
 * Every method invocation (on both StreamServiceRegistrar and 
ServiceRegistration)
 * is atomic with respect to other invocations.
 *
 * The StreamServiceRegistrar is intended to perform the same function
 * as the ServiceRegistrar, but with the ability to return results as a
 * stream, so memory consumption can be minimised at the client.
 *
 * All clients utilising ServiceRegistrar, should switch to the
 * StreamServiceRegistrar.
 *
 * @see ServiceRegistrar
 * @see PortableServiceRegistrar
 * @see ServiceRegistration
 * @author Peter Firmstone
 * @since 2.2.0
 */
public interface StreamServiceRegistrar extends ServiceRegistrar{

    /**
     * Returns a ResultStream that provides access to MarshalledServiceItem
     * instances.  The ResultStream terminates with a null value.  The 
result
     * stream may be infinite.
     *
     * MarshalledServiceItem extends ServiceItem and can be used anywhere a
     * ServiceItem can.  A MarshalledServiceItem implementation instance
     * contains the marshalled form of a Service and it's Entry's,
     * the corresponding superclass ServiceItem however contains null values
     * for the service and excludes any Entry's that are not 
specifically requested
     * unmarshalled by this method. The ServiceID will be unmarshalled 
always.
     *
     * This method is designed to allow the caller to control exactly what
     * is unmarshalled and when, it allows unmarshalling of specific entries
     * that the caller may wish to utilise for filtering. It is
     * designed to allow both the caller and the implementer to deal 
with very
     * large result sets in an incremental fashion.
     *
     * It is absolutely essential that the caller closes and deletes any 
references to
     * the returned result stream as soon as it is no longer requried.
     *
     * @param tmpl template to match
     * specified template
     * @param unmarshalledEntries only Entry's with these classes will be in
     * unmarshalled form.
     * @param maxBatchSize Allows the caller to limit the number of results
     * held locally, larger batch sizes reduce network traffic, but may 
delay
     * processing locally depending on implementation.
     * @return ResultStream containing ServiceItem's
     * @throws java.rmi.RemoteException
     * @see MarshalledServiceItem
     * @see ServiceItem
     * @see ResultStream
     * @see ServiceResultStreamFilter
     * @see ResultStreamUnmarshaller
     * @since 2.2.0
     */
    ResultStream lookup(ServiceTemplate tmpl,
        Class[] unmarshalledEntries, int maxBatchSize) throws IOException;
   
    /**
     * Looks at all service items that match the specified template, finds
     * every entry (among those service items) that either doesn't match any
     * entry templates or is a subclass of at least one matching entry
     * template, and returns the set of the (most specific) classes of those
     * entries.  Duplicate classes are eliminated, and the order of classes
     * within the returned array is arbitrary.  Null (not an empty array) is
     * returned if there are no such entries or no matching items.  If a
     * returned class cannot be deserialized, that element of the returned
     * array is set to null and no exception is thrown.
     *
     * @param tmpl template to match
     * @param maxBatchSize
     * @return a ResultStream containing Class of entry (attribute sets) 
for every service
     * that matches the specified template
     * @throws java.rmi.RemoteException
     */
    ResultStream getEntryClasses(ServiceTemplate tmpl, int maxBatchSize)
            throws IOException;

    /**
     * Looks at all service items that match the specified template, finds
     * every entry (among those service items) that matches
     * tmpl.attributeSetTemplates[setIndex], and returns the set of values
     * of the specified field of those entries.  Duplicate values are
     * eliminated, and the order of values isarbitrary. 
     * If a returned value cannot be deserialized, that
     * element is excluded and no exception is thrown.
     *
     * @param tmpl template to match
     * @param setIndex index into tmpl.attributeSetTemplates
     * @param field name of field of tmpl.attributeSetTemplates[setIndex]
     *
     * @param maxBatchSize
     * @return a ResultStream of objects that represents field values of 
entries
     * associated with services that meet the specified matching
     * criteria
     *
     * @throws NoSuchFieldException field does not name a field of the
     * entry template
     * @throws java.rmi.RemoteException
     */
    ResultStream getFieldValues(ServiceTemplate tmpl, int setIndex, 
String field,
            int maxBatchSize) throws NoSuchFieldException, IOException;
   
    /**
     * Looks at all service items that match the specified template, and for
     * every service item finds the most specific type (class or interface)
     * or types the service item is an instance of that are neither 
equal to,
     * nor a superclass of, any of the service types in the template and 
that
     * have names that start with the specified prefix, and returns the set
     * of all such types.  Duplicate types are eliminated, and the order of
     * types within the returned array is arbitrary. 
     * Null is returned if there are no such types.  If a returned type
     * cannot be deserialized, that element is excluded and no exception 
is thrown.
     *
     * @param tmpl template to match
     * @param prefix class name prefix
     *
     * @param maxBatchSize
     * @return a ResultStream containing a Class for all services that 
either match the
     * specified template or match the specified prefix
     * @throws java.rmi.RemoteException
     */
    ResultStream getServiceTypes(ServiceTemplate tmpl, String prefix,
            int maxBatchSize) throws IOException;

}

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

On 2/1/2011 2:30 PM, Christopher Dolan wrote:
> My alternative solution to the same problem as Gregg's was to compute
> SHA-1 hashes of the codebase jars, use that hash in the codebase URLs,
> and aggressively cache the jars client-side via a custom
> URLStreamHandlerFactory implementation instantiated in my
> PreferredClassProvider subclass.  It works great -- it dramatically
> reduced startup time after the first deployment and I no longer have to
> worry too much about codebase jar sizes because I almost never need to
> download them.  And it didn't require any code changes except in the
> createLoader() method of my PreferredClassProvider subclass.

Even that first "deployment", for me, could not start downloading 100s of jar 
files.  I did my vhttp: protocol handler for caching first, and it worked for my 
"small" early client deployments where only a single client interface was 
started and it would see 2-3 instances of the same service that it would list 
and let you pick one to access.  The slow start up was still disliked and I 
heard about the issue from many techs.  Even with a progress bar showing the 
downloading, indicating why it was not starting yet, they just said, we need to 
use the tool, not wait for it to download.  I really wanted to use mobile code 
though because it make a tremendous difference in the difficulty of deploying 
bug fixes and updates.  As picky as these people are, getting them to download 
and install an update was equally problematic.

In subsequent stages of evolution to the platform, I started adding a whole 
group of services/remote tools using Jini based discovery, so they could find an 
instance which was reachable, at one of multiple deployment locations.  I needed 
to put all the tools into one "installed package" and a ServiceUI desktop was a 
great way to integrate access to all services.

But, now instead of 3 codebases to resolve, there were 30 or more (10-20 per 
machine).  I just could not ask my customer to download all of that.  I even 
pre-populated my vhttp: protocol handlers cache as one form of non-initial 
download handling.  In the end, eliminating downloading until the service was 
requested was the only "complete" solution for me.  Everything else just felt 
like a bandaid, where as this solution feels complete to me because it addresses 
the exact problem by not doing what is unnecessary.  If I don't need to have a 
definition of a class from a remote server, then don't go there.

I'm glad to hear more about peoples customization of the service activation 
steps to learn how different solutions to the same problems I've had, have 
worked out.

Gregg

> Chris
>
> -----Original Message-----
> From: Gregg Wonderly [mailto:gregg@wonderly.org]
> Sent: Monday, January 31, 2011 7:30 PM
> To: river-dev@incubator.apache.org
> Subject: Re: MarshalledServiceItem
>
> One of the important things for my use of my changes to the
> ServiceRegistrar interface was separating deserialization of the Entrys
> from the service object.  Ultimately, I wanted to not have to worry
> about how the "codebase" was structured in order to minimize
> downloading. Some Jini deployers have done things to put a small jar at
> the front of the codebase that just has the preferred.list in it.  Even
> that download, for me, would have been too much.  My customers'
> deployments have nearly 50 codebases visible when my client starts up.
> Those 50 connections over a cellular or other high latency network would
> make service selection take way too long.
>
> Being able to defer downloading was also controlled by the changes to
> ClassLoading to include the "neverPrefer" settings on a class name.  I
> can do that now through the recent RMIClassLoaderSPI override
> capabilities.  But I still need to be able to control which Entry is
> deserialized.  I need to be able to ask what classes, by name, are in
> the namespace of each Entry object as my implementation allowed. I had
> made changes to Reggie marshalling to use reflection to get the class
> hierarchy, and then provided access to the list of classnames as part of
> the returned, marshalled value in my API.
>
> Gregg Wonderly
>
> On Jan 30, 2011, at 2:23 PM, Peter Firmstone wrote:
>
>> Gut feel, tells me the following class is not right, it's intent is to
> provide an api for implementation of delayed unmarshalling and
> provisioning of codebases, for services implementing a lookup service or
> ServiceRegistrar, whilst retaining backward compatibility.
>>
>> In other words, the lookup service proxy would implement it.
>>
>> Thoughts I had were to declare IOException's on methods, and to return
> a String annotation, rather than a URI.
>>
>> What are your thoughts?
>>
>> Cheers,
>>
>> Peter.
>>
>> /*
>> * Licensed to the Apache Software Foundation (ASF) under one
>> * or more contributor license agreements.  See the NOTICE file
>> * distributed with this work for additional information
>> * regarding copyright ownership. The ASF licenses this file
>> * to you under the Apache License, Version 2.0 (the
>> * "License"); you may not use this file except in compliance
>> * with the License. You may obtain a copy of the License at
>> *
>> *      http://www.apache.org/licenses/LICENSE-2.0
>> *
>> * Unless required by applicable law or agreed to in writing, software
>> * distributed under the License is distributed on an "AS IS" BASIS,
>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
>> * See the License for the specific language governing permissions and
>> * limitations under the License.
>> */
>>
>> package org.apache.river.api.lookup;
>>
>> import java.net.URI;
>> import java.security.CodeSource;
>> import net.jini.core.entry.Entry;
>> import net.jini.core.lookup.ServiceID;
>> import net.jini.core.lookup.ServiceItem;
>>
>> /**
>> * MarshalledServiceItem extends ServiceItem and can be used anywhere a
>> * ServiceItem can.  A MarshalledServiceItem implementation instance
>> * contains the marshalled form of a Service and it's Entry's,
>> * the corresponding superclass ServiceItem however contains null
> values
>> * for the service and can exclude any Entry's, however where Entry
>> * classes already exist at the client, that they be unmarshalled.
>> *
>> * The ServiceID shall be in unmarshalled form always in the
> ServiceItem super class.
>> *
>> * Since the ServiceItem.service is null, use of this class in existing
> software
>> * will not return the service, however it will not break that software
> as
>> * ServiceItem's contract is to set service or Entry's to null when
> they cannot
>> * be unmarshalled.
>> *
>> * ServiceItem's toString() method will return a different result for
>> * MarshalledServiceItem instances.
>> *
>> * If required, a new ServiceItem that is fully unmarshalled
>> * can be constructed from this class's methods and ServiceID.
>> *
>> * @author Peter Firmstone.
>> */
>> public abstract class MarshalledServiceItem extends ServiceItem{
>>    private static final long SerialVersionUID = 1L;
>>    protected MarshalledServiceItem(ServiceID id, Entry[]
> unmarshalledEntries){
>>        super(id, (Object) null, unmarshalledEntries);
>>    }
>>    /**
>>     * Unmarshall the service proxy.
>>     * @param load service with local or existing CodeSource or null for
>>     * default.
>>     * @return the service proxy, null if class not found.
>>     */
>>    public abstract Object getService(CodeSource[] code);
>>    /**
>>     * Unmarshall the Entry's
>>     * @return array of Entry's, null entry in array for any class not
> found.
>>     */
>>    public abstract Entry[] getEntries();
>>      public abstract URI[] getAnnotations();
>> }
>>
>
>

RE: MarshalledServiceItem

[Christopher Dolan <ch...@avid.com>]

My alternative solution to the same problem as Gregg's was to compute
SHA-1 hashes of the codebase jars, use that hash in the codebase URLs,
and aggressively cache the jars client-side via a custom
URLStreamHandlerFactory implementation instantiated in my
PreferredClassProvider subclass.  It works great -- it dramatically
reduced startup time after the first deployment and I no longer have to
worry too much about codebase jar sizes because I almost never need to
download them.  And it didn't require any code changes except in the
createLoader() method of my PreferredClassProvider subclass.

Chris

-----Original Message-----
From: Gregg Wonderly [mailto:gregg@wonderly.org] 
Sent: Monday, January 31, 2011 7:30 PM
To: river-dev@incubator.apache.org
Subject: Re: MarshalledServiceItem

One of the important things for my use of my changes to the
ServiceRegistrar interface was separating deserialization of the Entrys
from the service object.  Ultimately, I wanted to not have to worry
about how the "codebase" was structured in order to minimize
downloading. Some Jini deployers have done things to put a small jar at
the front of the codebase that just has the preferred.list in it.  Even
that download, for me, would have been too much.  My customers'
deployments have nearly 50 codebases visible when my client starts up.
Those 50 connections over a cellular or other high latency network would
make service selection take way too long.

Being able to defer downloading was also controlled by the changes to
ClassLoading to include the "neverPrefer" settings on a class name.  I
can do that now through the recent RMIClassLoaderSPI override
capabilities.  But I still need to be able to control which Entry is
deserialized.  I need to be able to ask what classes, by name, are in
the namespace of each Entry object as my implementation allowed. I had
made changes to Reggie marshalling to use reflection to get the class
hierarchy, and then provided access to the list of classnames as part of
the returned, marshalled value in my API.

Gregg Wonderly

On Jan 30, 2011, at 2:23 PM, Peter Firmstone wrote:

> Gut feel, tells me the following class is not right, it's intent is to
provide an api for implementation of delayed unmarshalling and
provisioning of codebases, for services implementing a lookup service or
ServiceRegistrar, whilst retaining backward compatibility.
> 
> In other words, the lookup service proxy would implement it.
> 
> Thoughts I had were to declare IOException's on methods, and to return
a String annotation, rather than a URI.
> 
> What are your thoughts?
> 
> Cheers,
> 
> Peter.
> 
> /*
> * Licensed to the Apache Software Foundation (ASF) under one
> * or more contributor license agreements.  See the NOTICE file
> * distributed with this work for additional information
> * regarding copyright ownership. The ASF licenses this file
> * to you under the Apache License, Version 2.0 (the
> * "License"); you may not use this file except in compliance
> * with the License. You may obtain a copy of the License at
> *
> *      http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> 
> package org.apache.river.api.lookup;
> 
> import java.net.URI;
> import java.security.CodeSource;
> import net.jini.core.entry.Entry;
> import net.jini.core.lookup.ServiceID;
> import net.jini.core.lookup.ServiceItem;
> 
> /**
> * MarshalledServiceItem extends ServiceItem and can be used anywhere a
> * ServiceItem can.  A MarshalledServiceItem implementation instance
> * contains the marshalled form of a Service and it's Entry's,
> * the corresponding superclass ServiceItem however contains null
values
> * for the service and can exclude any Entry's, however where Entry
> * classes already exist at the client, that they be unmarshalled.
> *
> * The ServiceID shall be in unmarshalled form always in the
ServiceItem super class.
> *
> * Since the ServiceItem.service is null, use of this class in existing
software
> * will not return the service, however it will not break that software
as
> * ServiceItem's contract is to set service or Entry's to null when
they cannot
> * be unmarshalled.
> *
> * ServiceItem's toString() method will return a different result for
> * MarshalledServiceItem instances.
> *
> * If required, a new ServiceItem that is fully unmarshalled
> * can be constructed from this class's methods and ServiceID.
> *
> * @author Peter Firmstone.
> */
> public abstract class MarshalledServiceItem extends ServiceItem{
>   private static final long SerialVersionUID = 1L;
>   protected MarshalledServiceItem(ServiceID id, Entry[]
unmarshalledEntries){
>       super(id, (Object) null, unmarshalledEntries);
>   }
>   /**
>    * Unmarshall the service proxy.
>    * @param load service with local or existing CodeSource or null for
>    * default.
>    * @return the service proxy, null if class not found.
>    */
>   public abstract Object getService(CodeSource[] code);
>   /**
>    * Unmarshall the Entry's
>    * @return array of Entry's, null entry in array for any class not
found.
>    */
>   public abstract Entry[] getEntries();
>     public abstract URI[] getAnnotations();
> }
> 

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Peter Firmstone wrote:
>> I'm looking for the simplest thing that will work, then we can look at
>> additions/extensions. Why tackle complicated downloading/classloading 
>> if we
>> can just solve the problem with a simple API extension that gives enough
>> flexibility for the common case?
>>   
>
> The simplified MarshalledServiceItem should be capable of that, the 
> registrar implementation proxy simply doesn't unmarshall until requested.
>
> Cheers,
>
> Peter.
>>
>>  
>>> Hence the method:
>>>
>>>
>>> ResultStream lookup(ServiceTemplate tmpl, Class[] 
>>> unmarshalledEntries, int
>>> maxBatchSize) throws IOException;
>>>
>>> But once we have the service we want, we'll need to download the jar 
>>> files.
>>>
>>>
>>>     
>> Yep,no chance of avoiding that....
>>
>>
>>   
>
>

Currently StreamServiceRegistrar has four methods, these are very 
similar to existing methods in ServiceRegistrar, the difference is they 
return a ResultStream instead of array's, maxBatchSize is an indication 
to the registrar proxy, how many records to cache locally:

ResultStream lookup(ServiceTemplate tmpl, Class[] unmarshalledEntries, 
int maxBatchSize) throws IOException;

ResultStream getEntryClasses(ServiceTemplate tmpl, int maxBatchSize) 
throws IOException;

ResultStream getFieldValues(ServiceTemplate tmpl, int setIndex, String 
field, int maxBatchSize) throws NoSuchFieldException, IOException;

ResultStream getServiceTypes(ServiceTemplate tmpl, String prefix, int 
maxBatchSize) throws IOException;

Apart from lookup, the additional methods may be useful for querying, 
the reasoning is registrars on the internet could potentially have very 
large result sets.  However as Dennis has suggested, the simplest thing 
that will work is to only have the lookup method, which provides to the 
registrar proxy, the classes of the Entry's the client is interested in.

What do you think?

Cheers,

Peter.

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Dan Creswell wrote:
>>> :) Yes, deployment but why are we choosing to deploy with this setup. What
>>> does it imply about services when they share .jar files?
>>>
>>>
>>>       
>> Because it becomes difficult to manage with existing tools when a codebase
>> contains multiple jar files for multiple services.  I believe that the
>> Codebase service implementors managed to make deployment easier by
>> automating codebase annotations.
>>     
>
>
> Or we could build some new tools....
>
>
>   
>>     
>>>>  Again, I'm sitting here thinking as a deployer of services, if I want to
>>>>
>>>>
>>>>         
>>>>> widget around and consolidate .jars I can do that ahead of time and then
>>>>> tweak service codebases via config just prior to deploy.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> But it would be faster for the client to receive unconsolidated jar's,
>>>> since other services might use some of these jars also.
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> Not for Gregg who hates roundtrips and multiple discoveries of codebases
>>> and
>>> such.
>>>
>>>
>>>       
>> If we have the Entry jar files installed at the client or first download
>> the jar files for Entry's we need, using the getEntryClasses method, then we
>> can unmarshall only these entry's which we already have codebases for during
>> lookup, of course at some point we're going to have to download something,
>> but we can avoid downloading the codebases for services we don't need.
>>
>>
>>
>>     
>>>       
>>>> What matters is the proxy gets the correct class files, in its own
>>>> private
>>>> namespace that are not shared with other proxy's (except for service api,
>>>> which may include Entry's), but we can save duplicate downloads.
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> Can you explain your reasoning for saving duplicate downloads? I think you
>>> mean because in some cases service's could share a codebase which they can
>>> do under the current scheme of course. Note that having "correct class
>>> files" isn't IMHO a sufficient constraint, it has to be a "particular
>>> collection of specific implementations of classes and versions".
>>>
>>>
>>>       
>> Correct, but it might reduce the size of the download when we've got to
>> bite the bullet and download the service jar files, if we've already got
>> some of the necessary jar's cached locally.
>>     
>
>
>   
> Size of download is only one thing we need to worry about - latency....
>   

Very true,  I think the services most suited to low latency connections 
will be smart proxy's that communicate with a remote server but do most 
things locally.

>
>   
>>     
>>>
>>>       
>>>> Maven provisioning is interesting.
>>>>
>>>> This is why I'm interested to investigate separating jar file identity
>>>> from
>>>> location, to simplify deployment and redundancy.  I'm putting my thoughts
>>>> out on the list, to gather responses, to see if there's a better way.
>>>>
>>>>
>>>>         
>>> Okay, so I think this also impacts on the stuff being discussed with
>>> Gregg.
>>> Whilst there are some complimentary aspects there are some costly steps in
>>> there as well.
>>>
>>>
>>>       
>> The trick is to delay the costly step until we've decided which service
>> instance we want.
>>
>>
>>
>>     
>>>       
>>>>
>>>>         
>>>>>
>>>>>           
>>>>>> Jini's lookup service lack of AND / OR querying capability is due to
>>>>>> security, the avoidance of instantiating foreign objects.
>>>>>>
>>>>>> Delayed unmarshalling of the service proxy allows service entry's to be
>>>>>> compared as objects, without requiring a codebase download for the
>>>>>> proxy
>>>>>> if
>>>>>> it's not the service we want, so it's not quite just returning a
>>>>>> MarshalledInstance.  This should be done without compromising the good
>>>>>> security features of the existing lookup service.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> I don't follow - I could tweak ServiceItem to hold the proxy as a
>>>>> MarshalledInstance and still expose all other "service identifying
>>>>> information". That MarshalledInstance mightn't even immediately carry
>>>>> the
>>>>> proxy code, could still be on the server and pulled down at point the
>>>>> consumer actually wants the proxy. Feels like some simple
>>>>> interface/sub-classing.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> This is true for cases where the service types don't matter to the
>>>> client,
>>>> I think Gregg wanted to elimate all codebase downloads until he was sure
>>>> he
>>>> had the correct service. Use of a MarshalledInstance could be an
>>>> acceptable
>>>> compromise, if Entry's have their own codebase annotations.  How would
>>>> this
>>>> affect the lookup semantics if we're looking for particular service types
>>>> though?
>>>>
>>>>
>>>>         
>>> Entry's have to have their own codebase for all cases of clients that
>>> don't
>>> know about those Entry's. Any client that has built-in knowledge of those
>>> Entry's will have them available on the classpath by virtue of it's need
>>> to
>>> specify them in it's lookup search.
>>>
>>>
>>>       
>> Except when we call getEntryClasses for a ServiceTemplate and someone's
>> created a new Entry, then we need reflection to get the fields, but that's
>> probably a rare case anyway.
>>
>>
>>  Can you explain more about the service types question?
>>     
>>>       
>> Reggie stores the class types of the service (not class files), so that
>> Reggie can use a ServiceTemplate to retrieve marshalled service proxy's that
>> match the service type defined in the template.
>>
>>
>>
>>  If I'm looking for a particular type, I specify a particular interface.
>>     
>> Exactly, so if we use a MarshalledInstance when we register the proxy, the
>> only interface (or class in this case) a client can specify is Object or
>> MarshalledInstance.
>>
>>
>>     
> Ah, no. MarshalledInstance is the general contract we're talking not the
> implementation. There is nothing to stop us building a subclass or some
> other container that compacts the proxy away but also retains/generates
> sufficient information to allow matching on types.
>
> And in fact, we only need that contract for the client. Registration needn't
> follow that form. Client's also will have Entry types they wish to specify
> for searches on their classpath.
>
> So the codebase trick is asymmetric at least potentially - we do it for
> clients, not services that are registering....
>
>
>   
True, which made me think a service interface that extends 
ServiceRegistrar, is just a service, which clients can chose to utilise 
if they need it, or otherwise just ignore.

It could be similar to reggie with an additional interface, while fully 
supporting the existing ServiceRegistrar lookup semantics.

In fact Reggie already has most of the needed pieces.

>>   I'm
>>     
>>> guessing you mean a service with specific Entry's? A client after some
>>> specific set of Entry's will already know those via classpath. We could
>>> simply allow a client to say "I'm only interested in Services with these
>>> Entry's, return me all matches but only give me the following Entry types
>>> as
>>> part of the ServiceItem". This feels much closer to the original intent
>>> "stop stuff getting to a client that it's not interested in" than trying
>>> to
>>> "control in detail all aspects of download and intimately dig around in
>>> service implementations, including .jars, to do it".
>>>
>>>
>>>       
>> I don't think we should dig around in jars etc to do it, just package
>> Entry's separately so we don't have to.  By providing the class files for
>> the Entry's we want unmarshalled via the lookup call, we wouldn't need to
>> download their jar files.
>>
>>     
>
> Package them separately and put them on client classpaths and then filter.
> In this fashion client has limited knowledge and can protect itself from
> additional knowledge via download side-effects by expressing it's desire to
> only see types it explicitly searches for/considers. Quite simply, if the
> client doesn't have an Entry on its classpath it has next to no interest in
> seeing that stuff.
>   

Ok.

> I'm looking for the simplest thing that will work, then we can look at
> additions/extensions. Why tackle complicated downloading/classloading if we
> can just solve the problem with a simple API extension that gives enough
> flexibility for the common case?
>   

The simplified MarshalledServiceItem should be capable of that, the 
registrar implementation proxy simply doesn't unmarshall until requested.

Cheers,

Peter.
>
>   
>> Hence the method:
>>
>>
>> ResultStream lookup(ServiceTemplate tmpl, Class[] unmarshalledEntries, int
>> maxBatchSize) throws IOException;
>>
>> But once we have the service we want, we'll need to download the jar files.
>>
>>
>>     
> Yep,no chance of avoiding that....
>
>
>   

Re: MarshalledServiceItem

[Tom Hobbs <tv...@googlemail.com>]

I think Michal made a very good suggestion with the invention of a
*new* service type called something like "ServiceBrowserService".

(Good call, Michal!).

Reading the to-ing and fro-ing discussing this use of
MarshalledServiceItem and the use of the lookup service, it does
appear that we're trying to squeeze the LUS into doing something that
it possibly isn't designed to do.  As Gregg has demonstrated, it can
be done and made to work well, but what is the cost of the complexity
in the infra/code cleanliness/design/etc?

A client can use a LUS to get *any* SBS, which can return non-domain
specific details describing the available services.  (e.g. name :
String, type : Class[], service Id : String, groups : String[], etc).
Potentially clients can be deployed with all the JARs needed to
unmarshal a SBS (so no download needed there), they can then see
what's available view it's UI, and so download only becomes necessary
when the user uses a LUS and details from the SBS to choose their
service.

Maybe this SBS topic needs to be split off into it's own thread.

On Wed, Feb 2, 2011 at 11:00 AM, Dan Creswell <da...@gmail.com> wrote:
> On 2 February 2011 10:33, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>> Dan Creswell wrote:
>>
>>>
>>>>
>>>>> Clarification:  Separate the location and identity of a jar file or
>>>>>
>>>>>
>>>> archive containing class files:
>>>>
>>>> We have integrity constraints, which rely on message digests to confirm
>>>> files have not been tampered with.
>>>>
>>>> The message digest (although now considered a weak form of encryption),
>>>> is
>>>> the identity of a jar file, well we use this type of information to
>>>> identify
>>>> it's the jar we expect.
>>>>
>>>> For httpmd, we use a URL string annotation (which could be an IP address
>>>> and port, or a DNS hostname and port), a path and file name, followed by
>>>> the
>>>> message digest.
>>>>
>>>> The IP address or dns hostname , port and path represent the location of
>>>> the jar file, while the file name and message digest represent it's
>>>> identity.
>>>>
>>>> If we separate the location + port, it can be discovered using DNS-SRV
>>>> records, allowing redundant codebase servers, while the identity is
>>>> limited
>>>> to the file name and message digest.
>>>>
>>>> Then the RMI codebase property only needs to be a domain in which a
>>>> suitable codebase can be discovered and queried.
>>>>
>>>>
>>>>
>>>>
>>> Mmm, but the process of resolution on codebases slows things down for
>>> Gregg.
>>>
>>>
>>
>> Correct, Gregg avoids the process of resolution, until it's absolutely
>> necessary, while Chris, caches his jar files after downloading, so he only
>> has to do so once.  DNS resolution might take 10 seconds, so this is an
>> important point you've made.  A local caching DNS would be advantageous.
>>
>>
>>  I think we're gonna have to decide which profiles of machines we're going
>>> to
>>> target with what solutions. I'm not suggesting we try and build for all
>>> the
>>> different profiles more that we'll have to choose what we do and do not
>>> support.
>>>  But, if service proxy's are sharing jar files, what does that mean?
>>> Someone
>>>
>>>
>>>> somewhere chose to package them all together like that for some reason.
>>>>> What
>>>>> reason, what problem are they solving?
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Deployment reasons.
>>>>
>>>>
>>>>
>>>>
>>> :) Yes, deployment but why are we choosing to deploy with this setup. What
>>> does it imply about services when they share .jar files?
>>>
>>>
>>
>> Because it becomes difficult to manage with existing tools when a codebase
>> contains multiple jar files for multiple services.  I believe that the
>> Codebase service implementors managed to make deployment easier by
>> automating codebase annotations.
>
>
> Or we could build some new tools....
>
>
>>
>>
>>>
>>>>  Again, I'm sitting here thinking as a deployer of services, if I want to
>>>>
>>>>
>>>>> widget around and consolidate .jars I can do that ahead of time and then
>>>>> tweak service codebases via config just prior to deploy.
>>>>>
>>>>>
>>>>>
>>>>>
>>>> But it would be faster for the client to receive unconsolidated jar's,
>>>> since other services might use some of these jars also.
>>>>
>>>>
>>>>
>>>>
>>> Not for Gregg who hates roundtrips and multiple discoveries of codebases
>>> and
>>> such.
>>>
>>>
>>
>> If we have the Entry jar files installed at the client or first download
>> the jar files for Entry's we need, using the getEntryClasses method, then we
>> can unmarshall only these entry's which we already have codebases for during
>> lookup, of course at some point we're going to have to download something,
>> but we can avoid downloading the codebases for services we don't need.
>>
>>
>>
>>>
>>>
>>>> What matters is the proxy gets the correct class files, in its own
>>>> private
>>>> namespace that are not shared with other proxy's (except for service api,
>>>> which may include Entry's), but we can save duplicate downloads.
>>>>
>>>>
>>>>
>>>>
>>> Can you explain your reasoning for saving duplicate downloads? I think you
>>> mean because in some cases service's could share a codebase which they can
>>> do under the current scheme of course. Note that having "correct class
>>> files" isn't IMHO a sufficient constraint, it has to be a "particular
>>> collection of specific implementations of classes and versions".
>>>
>>>
>>
>> Correct, but it might reduce the size of the download when we've got to
>> bite the bullet and download the service jar files, if we've already got
>> some of the necessary jar's cached locally.
>
>
> Size of download is only one thing we need to worry about - latency....
>
>
>>
>>
>>>
>>>
>>>
>>>> Maven provisioning is interesting.
>>>>
>>>> This is why I'm interested to investigate separating jar file identity
>>>> from
>>>> location, to simplify deployment and redundancy.  I'm putting my thoughts
>>>> out on the list, to gather responses, to see if there's a better way.
>>>>
>>>>
>>>
>>>
>>> Okay, so I think this also impacts on the stuff being discussed with
>>> Gregg.
>>> Whilst there are some complimentary aspects there are some costly steps in
>>> there as well.
>>>
>>>
>>
>> The trick is to delay the costly step until we've decided which service
>> instance we want.
>>
>>
>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Jini's lookup service lack of AND / OR querying capability is due to
>>>>>> security, the avoidance of instantiating foreign objects.
>>>>>>
>>>>>> Delayed unmarshalling of the service proxy allows service entry's to be
>>>>>> compared as objects, without requiring a codebase download for the
>>>>>> proxy
>>>>>> if
>>>>>> it's not the service we want, so it's not quite just returning a
>>>>>> MarshalledInstance.  This should be done without compromising the good
>>>>>> security features of the existing lookup service.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> I don't follow - I could tweak ServiceItem to hold the proxy as a
>>>>> MarshalledInstance and still expose all other "service identifying
>>>>> information". That MarshalledInstance mightn't even immediately carry
>>>>> the
>>>>> proxy code, could still be on the server and pulled down at point the
>>>>> consumer actually wants the proxy. Feels like some simple
>>>>> interface/sub-classing.
>>>>>
>>>>>
>>>>>
>>>>>
>>>> This is true for cases where the service types don't matter to the
>>>> client,
>>>> I think Gregg wanted to elimate all codebase downloads until he was sure
>>>> he
>>>> had the correct service. Use of a MarshalledInstance could be an
>>>> acceptable
>>>> compromise, if Entry's have their own codebase annotations.  How would
>>>> this
>>>> affect the lookup semantics if we're looking for particular service types
>>>> though?
>>>>
>>>>
>>>
>>>
>>> Entry's have to have their own codebase for all cases of clients that
>>> don't
>>> know about those Entry's. Any client that has built-in knowledge of those
>>> Entry's will have them available on the classpath by virtue of it's need
>>> to
>>> specify them in it's lookup search.
>>>
>>>
>>
>> Except when we call getEntryClasses for a ServiceTemplate and someone's
>> created a new Entry, then we need reflection to get the fields, but that's
>> probably a rare case anyway.
>>
>>
>>  Can you explain more about the service types question?
>>>
>>>
>>
>> Reggie stores the class types of the service (not class files), so that
>> Reggie can use a ServiceTemplate to retrieve marshalled service proxy's that
>> match the service type defined in the template.
>>
>>
>>
>>  If I'm looking for a particular type, I specify a particular interface.
>>>
>>
>> Exactly, so if we use a MarshalledInstance when we register the proxy, the
>> only interface (or class in this case) a client can specify is Object or
>> MarshalledInstance.
>>
>>
> Ah, no. MarshalledInstance is the general contract we're talking not the
> implementation. There is nothing to stop us building a subclass or some
> other container that compacts the proxy away but also retains/generates
> sufficient information to allow matching on types.
>
> And in fact, we only need that contract for the client. Registration needn't
> follow that form. Client's also will have Entry types they wish to specify
> for searches on their classpath.
>
> So the codebase trick is asymmetric at least potentially - we do it for
> clients, not services that are registering....
>
>
>>   I'm
>>> guessing you mean a service with specific Entry's? A client after some
>>> specific set of Entry's will already know those via classpath. We could
>>> simply allow a client to say "I'm only interested in Services with these
>>> Entry's, return me all matches but only give me the following Entry types
>>> as
>>> part of the ServiceItem". This feels much closer to the original intent
>>> "stop stuff getting to a client that it's not interested in" than trying
>>> to
>>> "control in detail all aspects of download and intimately dig around in
>>> service implementations, including .jars, to do it".
>>>
>>>
>>
>> I don't think we should dig around in jars etc to do it, just package
>> Entry's separately so we don't have to.  By providing the class files for
>> the Entry's we want unmarshalled via the lookup call, we wouldn't need to
>> download their jar files.
>>
>
> Package them separately and put them on client classpaths and then filter.
> In this fashion client has limited knowledge and can protect itself from
> additional knowledge via download side-effects by expressing it's desire to
> only see types it explicitly searches for/considers. Quite simply, if the
> client doesn't have an Entry on its classpath it has next to no interest in
> seeing that stuff.
>
> I'm looking for the simplest thing that will work, then we can look at
> additions/extensions. Why tackle complicated downloading/classloading if we
> can just solve the problem with a simple API extension that gives enough
> flexibility for the common case?
>
>
>>
>> Hence the method:
>>
>>
>> ResultStream lookup(ServiceTemplate tmpl, Class[] unmarshalledEntries, int
>> maxBatchSize) throws IOException;
>>
>> But once we have the service we want, we'll need to download the jar files.
>>
>>
> Yep,no chance of avoiding that....
>
>
>>
>>
>>>
>>>
>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Over the internet, we could potentially have very large lookup
>>>>>> services,
>>>>>> by
>>>>>> allowing clients to remove unwanted services from their results before
>>>>>> unmarshalling, we can reduce the resources required of the client:
>>>>>>
>>>>>>  * Network Bandwidth, clients don't need to download unwanted
>>>>>> codebases.
>>>>>>  * Memory (ClassLoader and unwanted classes are not loaded into
>>>>>> memory).
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> If we were to go across internet have we squared away use of e.g.
>>>>> DNS-SD?
>>>>> i.e. Is it a given we'll expose a classic JINI LUS?
>>>>>
>>>>>
>>>>>
>>>>>
>>>> The semantics of DNS-SD make it well suited to discovery of a lookup
>>>> service.  I think Sim highlighted earlier that its difficult to get
>>>> domain
>>>> administrators to do Dynamic Updated DNS-SD, they're comfortable with
>>>> DNS-SRV records, so it would appear easier to rely on DNS-SD as a lookup
>>>> locator / domain browser, not as a lookup service replacement.
>>>>
>>>>
>>>>
>>>
>>> Agreed. Solves my concerns of exposing unicast locators of the traditional
>>> type and multicast across the net.
>>>
>>>
>>>
>>>
>> ;)
>>
>> Cheers,
>>
>> Peter.
>>
>

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

On 2 February 2011 10:33, Peter Firmstone <ji...@zeus.net.au> wrote:

> Dan Creswell wrote:
>
>>
>>>
>>>> Clarification:  Separate the location and identity of a jar file or
>>>>
>>>>
>>> archive containing class files:
>>>
>>> We have integrity constraints, which rely on message digests to confirm
>>> files have not been tampered with.
>>>
>>> The message digest (although now considered a weak form of encryption),
>>> is
>>> the identity of a jar file, well we use this type of information to
>>> identify
>>> it's the jar we expect.
>>>
>>> For httpmd, we use a URL string annotation (which could be an IP address
>>> and port, or a DNS hostname and port), a path and file name, followed by
>>> the
>>> message digest.
>>>
>>> The IP address or dns hostname , port and path represent the location of
>>> the jar file, while the file name and message digest represent it's
>>> identity.
>>>
>>> If we separate the location + port, it can be discovered using DNS-SRV
>>> records, allowing redundant codebase servers, while the identity is
>>> limited
>>> to the file name and message digest.
>>>
>>> Then the RMI codebase property only needs to be a domain in which a
>>> suitable codebase can be discovered and queried.
>>>
>>>
>>>
>>>
>> Mmm, but the process of resolution on codebases slows things down for
>> Gregg.
>>
>>
>
> Correct, Gregg avoids the process of resolution, until it's absolutely
> necessary, while Chris, caches his jar files after downloading, so he only
> has to do so once.  DNS resolution might take 10 seconds, so this is an
> important point you've made.  A local caching DNS would be advantageous.
>
>
>  I think we're gonna have to decide which profiles of machines we're going
>> to
>> target with what solutions. I'm not suggesting we try and build for all
>> the
>> different profiles more that we'll have to choose what we do and do not
>> support.
>>  But, if service proxy's are sharing jar files, what does that mean?
>> Someone
>>
>>
>>> somewhere chose to package them all together like that for some reason.
>>>> What
>>>> reason, what problem are they solving?
>>>>
>>>>
>>>>
>>>>
>>> Deployment reasons.
>>>
>>>
>>>
>>>
>> :) Yes, deployment but why are we choosing to deploy with this setup. What
>> does it imply about services when they share .jar files?
>>
>>
>
> Because it becomes difficult to manage with existing tools when a codebase
> contains multiple jar files for multiple services.  I believe that the
> Codebase service implementors managed to make deployment easier by
> automating codebase annotations.


Or we could build some new tools....


>
>
>>
>>>  Again, I'm sitting here thinking as a deployer of services, if I want to
>>>
>>>
>>>> widget around and consolidate .jars I can do that ahead of time and then
>>>> tweak service codebases via config just prior to deploy.
>>>>
>>>>
>>>>
>>>>
>>> But it would be faster for the client to receive unconsolidated jar's,
>>> since other services might use some of these jars also.
>>>
>>>
>>>
>>>
>> Not for Gregg who hates roundtrips and multiple discoveries of codebases
>> and
>> such.
>>
>>
>
> If we have the Entry jar files installed at the client or first download
> the jar files for Entry's we need, using the getEntryClasses method, then we
> can unmarshall only these entry's which we already have codebases for during
> lookup, of course at some point we're going to have to download something,
> but we can avoid downloading the codebases for services we don't need.
>
>
>
>>
>>
>>> What matters is the proxy gets the correct class files, in its own
>>> private
>>> namespace that are not shared with other proxy's (except for service api,
>>> which may include Entry's), but we can save duplicate downloads.
>>>
>>>
>>>
>>>
>> Can you explain your reasoning for saving duplicate downloads? I think you
>> mean because in some cases service's could share a codebase which they can
>> do under the current scheme of course. Note that having "correct class
>> files" isn't IMHO a sufficient constraint, it has to be a "particular
>> collection of specific implementations of classes and versions".
>>
>>
>
> Correct, but it might reduce the size of the download when we've got to
> bite the bullet and download the service jar files, if we've already got
> some of the necessary jar's cached locally.


Size of download is only one thing we need to worry about - latency....


>
>
>>
>>
>>
>>> Maven provisioning is interesting.
>>>
>>> This is why I'm interested to investigate separating jar file identity
>>> from
>>> location, to simplify deployment and redundancy.  I'm putting my thoughts
>>> out on the list, to gather responses, to see if there's a better way.
>>>
>>>
>>
>>
>> Okay, so I think this also impacts on the stuff being discussed with
>> Gregg.
>> Whilst there are some complimentary aspects there are some costly steps in
>> there as well.
>>
>>
>
> The trick is to delay the costly step until we've decided which service
> instance we want.
>
>
>
>>
>>
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>> Jini's lookup service lack of AND / OR querying capability is due to
>>>>> security, the avoidance of instantiating foreign objects.
>>>>>
>>>>> Delayed unmarshalling of the service proxy allows service entry's to be
>>>>> compared as objects, without requiring a codebase download for the
>>>>> proxy
>>>>> if
>>>>> it's not the service we want, so it's not quite just returning a
>>>>> MarshalledInstance.  This should be done without compromising the good
>>>>> security features of the existing lookup service.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> I don't follow - I could tweak ServiceItem to hold the proxy as a
>>>> MarshalledInstance and still expose all other "service identifying
>>>> information". That MarshalledInstance mightn't even immediately carry
>>>> the
>>>> proxy code, could still be on the server and pulled down at point the
>>>> consumer actually wants the proxy. Feels like some simple
>>>> interface/sub-classing.
>>>>
>>>>
>>>>
>>>>
>>> This is true for cases where the service types don't matter to the
>>> client,
>>> I think Gregg wanted to elimate all codebase downloads until he was sure
>>> he
>>> had the correct service. Use of a MarshalledInstance could be an
>>> acceptable
>>> compromise, if Entry's have their own codebase annotations.  How would
>>> this
>>> affect the lookup semantics if we're looking for particular service types
>>> though?
>>>
>>>
>>
>>
>> Entry's have to have their own codebase for all cases of clients that
>> don't
>> know about those Entry's. Any client that has built-in knowledge of those
>> Entry's will have them available on the classpath by virtue of it's need
>> to
>> specify them in it's lookup search.
>>
>>
>
> Except when we call getEntryClasses for a ServiceTemplate and someone's
> created a new Entry, then we need reflection to get the fields, but that's
> probably a rare case anyway.
>
>
>  Can you explain more about the service types question?
>>
>>
>
> Reggie stores the class types of the service (not class files), so that
> Reggie can use a ServiceTemplate to retrieve marshalled service proxy's that
> match the service type defined in the template.
>
>
>
>  If I'm looking for a particular type, I specify a particular interface.
>>
>
> Exactly, so if we use a MarshalledInstance when we register the proxy, the
> only interface (or class in this case) a client can specify is Object or
> MarshalledInstance.
>
>
Ah, no. MarshalledInstance is the general contract we're talking not the
implementation. There is nothing to stop us building a subclass or some
other container that compacts the proxy away but also retains/generates
sufficient information to allow matching on types.

And in fact, we only need that contract for the client. Registration needn't
follow that form. Client's also will have Entry types they wish to specify
for searches on their classpath.

So the codebase trick is asymmetric at least potentially - we do it for
clients, not services that are registering....


>   I'm
>> guessing you mean a service with specific Entry's? A client after some
>> specific set of Entry's will already know those via classpath. We could
>> simply allow a client to say "I'm only interested in Services with these
>> Entry's, return me all matches but only give me the following Entry types
>> as
>> part of the ServiceItem". This feels much closer to the original intent
>> "stop stuff getting to a client that it's not interested in" than trying
>> to
>> "control in detail all aspects of download and intimately dig around in
>> service implementations, including .jars, to do it".
>>
>>
>
> I don't think we should dig around in jars etc to do it, just package
> Entry's separately so we don't have to.  By providing the class files for
> the Entry's we want unmarshalled via the lookup call, we wouldn't need to
> download their jar files.
>

Package them separately and put them on client classpaths and then filter.
In this fashion client has limited knowledge and can protect itself from
additional knowledge via download side-effects by expressing it's desire to
only see types it explicitly searches for/considers. Quite simply, if the
client doesn't have an Entry on its classpath it has next to no interest in
seeing that stuff.

I'm looking for the simplest thing that will work, then we can look at
additions/extensions. Why tackle complicated downloading/classloading if we
can just solve the problem with a simple API extension that gives enough
flexibility for the common case?


>
> Hence the method:
>
>
> ResultStream lookup(ServiceTemplate tmpl, Class[] unmarshalledEntries, int
> maxBatchSize) throws IOException;
>
> But once we have the service we want, we'll need to download the jar files.
>
>
Yep,no chance of avoiding that....


>
>
>>
>>
>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>> Over the internet, we could potentially have very large lookup
>>>>> services,
>>>>> by
>>>>> allowing clients to remove unwanted services from their results before
>>>>> unmarshalling, we can reduce the resources required of the client:
>>>>>
>>>>>  * Network Bandwidth, clients don't need to download unwanted
>>>>> codebases.
>>>>>  * Memory (ClassLoader and unwanted classes are not loaded into
>>>>> memory).
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> If we were to go across internet have we squared away use of e.g.
>>>> DNS-SD?
>>>> i.e. Is it a given we'll expose a classic JINI LUS?
>>>>
>>>>
>>>>
>>>>
>>> The semantics of DNS-SD make it well suited to discovery of a lookup
>>> service.  I think Sim highlighted earlier that its difficult to get
>>> domain
>>> administrators to do Dynamic Updated DNS-SD, they're comfortable with
>>> DNS-SRV records, so it would appear easier to rely on DNS-SD as a lookup
>>> locator / domain browser, not as a lookup service replacement.
>>>
>>>
>>>
>>
>> Agreed. Solves my concerns of exposing unicast locators of the traditional
>> type and multicast across the net.
>>
>>
>>
>>
> ;)
>
> Cheers,
>
> Peter.
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Dan Creswell wrote:
>>     
>>> Clarification:  Separate the location and identity of a jar file or
>>>       
>> archive containing class files:
>>
>> We have integrity constraints, which rely on message digests to confirm
>> files have not been tampered with.
>>
>> The message digest (although now considered a weak form of encryption), is
>> the identity of a jar file, well we use this type of information to identify
>> it's the jar we expect.
>>
>> For httpmd, we use a URL string annotation (which could be an IP address
>> and port, or a DNS hostname and port), a path and file name, followed by the
>> message digest.
>>
>> The IP address or dns hostname , port and path represent the location of
>> the jar file, while the file name and message digest represent it's
>> identity.
>>
>> If we separate the location + port, it can be discovered using DNS-SRV
>> records, allowing redundant codebase servers, while the identity is limited
>> to the file name and message digest.
>>
>> Then the RMI codebase property only needs to be a domain in which a
>> suitable codebase can be discovered and queried.
>>
>>
>>     
> Mmm, but the process of resolution on codebases slows things down for Gregg.
>   

Correct, Gregg avoids the process of resolution, until it's absolutely 
necessary, while Chris, caches his jar files after downloading, so he 
only has to do so once.  DNS resolution might take 10 seconds, so this 
is an important point you've made.  A local caching DNS would be 
advantageous.

> I think we're gonna have to decide which profiles of machines we're going to
> target with what solutions. I'm not suggesting we try and build for all the
> different profiles more that we'll have to choose what we do and do not
> support.
>   
> But, if service proxy's are sharing jar files, what does that mean? Someone
>   
>>> somewhere chose to package them all together like that for some reason.
>>> What
>>> reason, what problem are they solving?
>>>
>>>
>>>       
>> Deployment reasons.
>>
>>
>>     
> :) Yes, deployment but why are we choosing to deploy with this setup. What
> does it imply about services when they share .jar files?
>   

Because it becomes difficult to manage with existing tools when a 
codebase contains multiple jar files for multiple services.  I believe 
that the Codebase service implementors managed to make deployment easier 
by automating codebase annotations.
>   
>>  Again, I'm sitting here thinking as a deployer of services, if I want to
>>     
>>> widget around and consolidate .jars I can do that ahead of time and then
>>> tweak service codebases via config just prior to deploy.
>>>
>>>
>>>       
>> But it would be faster for the client to receive unconsolidated jar's,
>> since other services might use some of these jars also.
>>
>>
>>     
> Not for Gregg who hates roundtrips and multiple discoveries of codebases and
> such.
>   

If we have the Entry jar files installed at the client or first download 
the jar files for Entry's we need, using the getEntryClasses method, 
then we can unmarshall only these entry's which we already have 
codebases for during lookup, of course at some point we're going to have 
to download something, but we can avoid downloading the codebases for 
services we don't need.

>
>   
>> What matters is the proxy gets the correct class files, in its own private
>> namespace that are not shared with other proxy's (except for service api,
>> which may include Entry's), but we can save duplicate downloads.
>>
>>
>>     
> Can you explain your reasoning for saving duplicate downloads? I think you
> mean because in some cases service's could share a codebase which they can
> do under the current scheme of course. Note that having "correct class
> files" isn't IMHO a sufficient constraint, it has to be a "particular
> collection of specific implementations of classes and versions".
>   

Correct, but it might reduce the size of the download when we've got to 
bite the bullet and download the service jar files, if we've already got 
some of the necessary jar's cached locally.
>
>
>   
>> Maven provisioning is interesting.
>>
>> This is why I'm interested to investigate separating jar file identity from
>> location, to simplify deployment and redundancy.  I'm putting my thoughts
>> out on the list, to gather responses, to see if there's a better way.
>>     
>
>
> Okay, so I think this also impacts on the stuff being discussed with Gregg.
> Whilst there are some complimentary aspects there are some costly steps in
> there as well.
>   

The trick is to delay the costly step until we've decided which service 
instance we want.

>
>   
>>
>>     
>>>
>>>       
>>>> Jini's lookup service lack of AND / OR querying capability is due to
>>>> security, the avoidance of instantiating foreign objects.
>>>>
>>>> Delayed unmarshalling of the service proxy allows service entry's to be
>>>> compared as objects, without requiring a codebase download for the proxy
>>>> if
>>>> it's not the service we want, so it's not quite just returning a
>>>> MarshalledInstance.  This should be done without compromising the good
>>>> security features of the existing lookup service.
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> I don't follow - I could tweak ServiceItem to hold the proxy as a
>>> MarshalledInstance and still expose all other "service identifying
>>> information". That MarshalledInstance mightn't even immediately carry the
>>> proxy code, could still be on the server and pulled down at point the
>>> consumer actually wants the proxy. Feels like some simple
>>> interface/sub-classing.
>>>
>>>
>>>       
>> This is true for cases where the service types don't matter to the client,
>> I think Gregg wanted to elimate all codebase downloads until he was sure he
>> had the correct service. Use of a MarshalledInstance could be an acceptable
>> compromise, if Entry's have their own codebase annotations.  How would this
>> affect the lookup semantics if we're looking for particular service types
>> though?
>>     
>
>
> Entry's have to have their own codebase for all cases of clients that don't
> know about those Entry's. Any client that has built-in knowledge of those
> Entry's will have them available on the classpath by virtue of it's need to
> specify them in it's lookup search.
>   

Except when we call getEntryClasses for a ServiceTemplate and someone's 
created a new Entry, then we need reflection to get the fields, but 
that's probably a rare case anyway.

> Can you explain more about the service types question?
>   

Reggie stores the class types of the service (not class files), so that 
Reggie can use a ServiceTemplate to retrieve marshalled service proxy's 
that match the service type defined in the template.


> If I'm looking for a particular type, I specify a particular interface.

Exactly, so if we use a MarshalledInstance when we register the proxy, 
the only interface (or class in this case) a client can specify is 
Object or MarshalledInstance.

>  I'm
> guessing you mean a service with specific Entry's? A client after some
> specific set of Entry's will already know those via classpath. We could
> simply allow a client to say "I'm only interested in Services with these
> Entry's, return me all matches but only give me the following Entry types as
> part of the ServiceItem". This feels much closer to the original intent
> "stop stuff getting to a client that it's not interested in" than trying to
> "control in detail all aspects of download and intimately dig around in
> service implementations, including .jars, to do it".
>   

I don't think we should dig around in jars etc to do it, just package 
Entry's separately so we don't have to.  By providing the class files 
for the Entry's we want unmarshalled via the lookup call, we wouldn't 
need to download their jar files.

Hence the method:

ResultStream lookup(ServiceTemplate tmpl, Class[] unmarshalledEntries, 
int maxBatchSize) throws IOException;

But once we have the service we want, we'll need to download the jar files.

>
>
>   
>>     
>>>
>>>       
>>>> Over the internet, we could potentially have very large lookup services,
>>>> by
>>>> allowing clients to remove unwanted services from their results before
>>>> unmarshalling, we can reduce the resources required of the client:
>>>>
>>>>  * Network Bandwidth, clients don't need to download unwanted codebases.
>>>>  * Memory (ClassLoader and unwanted classes are not loaded into memory).
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> If we were to go across internet have we squared away use of e.g. DNS-SD?
>>> i.e. Is it a given we'll expose a classic JINI LUS?
>>>
>>>
>>>       
>> The semantics of DNS-SD make it well suited to discovery of a lookup
>> service.  I think Sim highlighted earlier that its difficult to get domain
>> administrators to do Dynamic Updated DNS-SD, they're comfortable with
>> DNS-SRV records, so it would appear easier to rely on DNS-SD as a lookup
>> locator / domain browser, not as a lookup service replacement.
>>
>>     
>
> Agreed. Solves my concerns of exposing unicast locators of the traditional
> type and multicast across the net.
>
>
>   
;)

Cheers,

Peter.

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

>
>
>> Clarification:  Separate the location and identity of a jar file or
> archive containing class files:
>
> We have integrity constraints, which rely on message digests to confirm
> files have not been tampered with.
>
> The message digest (although now considered a weak form of encryption), is
> the identity of a jar file, well we use this type of information to identify
> it's the jar we expect.
>
> For httpmd, we use a URL string annotation (which could be an IP address
> and port, or a DNS hostname and port), a path and file name, followed by the
> message digest.
>
> The IP address or dns hostname , port and path represent the location of
> the jar file, while the file name and message digest represent it's
> identity.
>
> If we separate the location + port, it can be discovered using DNS-SRV
> records, allowing redundant codebase servers, while the identity is limited
> to the file name and message digest.
>
> Then the RMI codebase property only needs to be a domain in which a
> suitable codebase can be discovered and queried.
>
>
Mmm, but the process of resolution on codebases slows things down for Gregg.
I think we're gonna have to decide which profiles of machines we're going to
target with what solutions. I'm not suggesting we try and build for all the
different profiles more that we'll have to choose what we do and do not
support.

But, if service proxy's are sharing jar files, what does that mean? Someone
>> somewhere chose to package them all together like that for some reason.
>> What
>> reason, what problem are they solving?
>>
>>
> Deployment reasons.
>
>
:) Yes, deployment but why are we choosing to deploy with this setup. What
does it imply about services when they share .jar files?

>
>  Again, I'm sitting here thinking as a deployer of services, if I want to
>> widget around and consolidate .jars I can do that ahead of time and then
>> tweak service codebases via config just prior to deploy.
>>
>>
>
> But it would be faster for the client to receive unconsolidated jar's,
> since other services might use some of these jars also.
>
>
Not for Gregg who hates roundtrips and multiple discoveries of codebases and
such.


> What matters is the proxy gets the correct class files, in its own private
> namespace that are not shared with other proxy's (except for service api,
> which may include Entry's), but we can save duplicate downloads.
>
>
Can you explain your reasoning for saving duplicate downloads? I think you
mean because in some cases service's could share a codebase which they can
do under the current scheme of course. Note that having "correct class
files" isn't IMHO a sufficient constraint, it has to be a "particular
collection of specific implementations of classes and versions".



> Maven provisioning is interesting.
>
> This is why I'm interested to investigate separating jar file identity from
> location, to simplify deployment and redundancy.  I'm putting my thoughts
> out on the list, to gather responses, to see if there's a better way.


Okay, so I think this also impacts on the stuff being discussed with Gregg.
Whilst there are some complimentary aspects there are some costly steps in
there as well.


>
>
>
>>
>>
>>
>>> Jini's lookup service lack of AND / OR querying capability is due to
>>> security, the avoidance of instantiating foreign objects.
>>>
>>> Delayed unmarshalling of the service proxy allows service entry's to be
>>> compared as objects, without requiring a codebase download for the proxy
>>> if
>>> it's not the service we want, so it's not quite just returning a
>>> MarshalledInstance.  This should be done without compromising the good
>>> security features of the existing lookup service.
>>>
>>>
>>>
>>>
>> I don't follow - I could tweak ServiceItem to hold the proxy as a
>> MarshalledInstance and still expose all other "service identifying
>> information". That MarshalledInstance mightn't even immediately carry the
>> proxy code, could still be on the server and pulled down at point the
>> consumer actually wants the proxy. Feels like some simple
>> interface/sub-classing.
>>
>>
>
> This is true for cases where the service types don't matter to the client,
> I think Gregg wanted to elimate all codebase downloads until he was sure he
> had the correct service. Use of a MarshalledInstance could be an acceptable
> compromise, if Entry's have their own codebase annotations.  How would this
> affect the lookup semantics if we're looking for particular service types
> though?


Entry's have to have their own codebase for all cases of clients that don't
know about those Entry's. Any client that has built-in knowledge of those
Entry's will have them available on the classpath by virtue of it's need to
specify them in it's lookup search.

Can you explain more about the service types question?

If I'm looking for a particular type, I specify a particular interface. I'm
guessing you mean a service with specific Entry's? A client after some
specific set of Entry's will already know those via classpath. We could
simply allow a client to say "I'm only interested in Services with these
Entry's, return me all matches but only give me the following Entry types as
part of the ServiceItem". This feels much closer to the original intent
"stop stuff getting to a client that it's not interested in" than trying to
"control in detail all aspects of download and intimately dig around in
service implementations, including .jars, to do it".



>
>
>>
>>
>>
>>> Over the internet, we could potentially have very large lookup services,
>>> by
>>> allowing clients to remove unwanted services from their results before
>>> unmarshalling, we can reduce the resources required of the client:
>>>
>>>  * Network Bandwidth, clients don't need to download unwanted codebases.
>>>  * Memory (ClassLoader and unwanted classes are not loaded into memory).
>>>
>>>
>>>
>>>
>> If we were to go across internet have we squared away use of e.g. DNS-SD?
>> i.e. Is it a given we'll expose a classic JINI LUS?
>>
>>
>
> The semantics of DNS-SD make it well suited to discovery of a lookup
> service.  I think Sim highlighted earlier that its difficult to get domain
> administrators to do Dynamic Updated DNS-SD, they're comfortable with
> DNS-SRV records, so it would appear easier to rely on DNS-SD as a lookup
> locator / domain browser, not as a lookup service replacement.
>

Agreed. Solves my concerns of exposing unicast locators of the traditional
type and multicast across the net.


>
> Cheers,
>
> Peter.
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Dan Creswell wrote:
> Ah, I know Sim is gonna hate this but I feel the need to retain full context
> for now....
>
> On 1 February 2011 10:55, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> Dan,
>>
>> I've appended the updated class after our earlier discussion,
>> MarshalledServiceItem, now only has one method as suggested.
>>
>> An option is to consider giving Entry's their own jar files, and to
>> consider doing something about providing a new codebase URL Handler, for
>> separating identity from location, so we can discover the same jar files in
>> other locations,
>>     
>
>
> I don't think I understand the separation of identity from location fully
> yet so let's try a framing statement:
>
> A Service's ID and it's type plus some minimum set of Entrys to identify
> pertinent service details is enough isn't it?. And any client wishing to
> identify such a service would require those minimal Entry's (agreed it
> mightn't want the others, could filter that at Lookup Service side such that
> it never gets to the client thus doesn't need downloading).
>
> I don't see the need to play marshalling games so much as delay opening up
> the proxy and limiting the Entry's a client sees to those it declares
> interest in.
>   

Clarification:  Separate the location and identity of a jar file or 
archive containing class files:

We have integrity constraints, which rely on message digests to confirm 
files have not been tampered with.

The message digest (although now considered a weak form of encryption), 
is the identity of a jar file, well we use this type of information to 
identify it's the jar we expect.

For httpmd, we use a URL string annotation (which could be an IP address 
and port, or a DNS hostname and port), a path and file name, followed by 
the message digest.

The IP address or dns hostname , port and path represent the location of 
the jar file, while the file name and message digest represent it's 
identity.

If we separate the location + port, it can be discovered using DNS-SRV 
records, allowing redundant codebase servers, while the identity is 
limited to the file name and message digest.

Then the RMI codebase property only needs to be a domain in which a 
suitable codebase can be discovered and queried.


>
>   
>> similar to Codebase Services with message digests or Maven Provisioning as
>> suggested by Dennis.  This doesn't mean I don't support the idea that the
>> best location for a codebase is at the originating service node, just that
>> it's likely that many service proxy's could share identical jar files, so
>> why should the client re-download something it already has?
>>
>>
>>     
> But, if service proxy's are sharing jar files, what does that mean? Someone
> somewhere chose to package them all together like that for some reason. What
> reason, what problem are they solving?
>   
Deployment reasons.

> Again, I'm sitting here thinking as a deployer of services, if I want to
> widget around and consolidate .jars I can do that ahead of time and then
> tweak service codebases via config just prior to deploy.
>   

But it would be faster for the client to receive unconsolidated jar's, 
since other services might use some of these jars also.

What matters is the proxy gets the correct class files, in its own 
private namespace that are not shared with other proxy's (except for 
service api, which may include Entry's), but we can save duplicate 
downloads.

Maven provisioning is interesting.

This is why I'm interested to investigate separating jar file identity 
from location, to simplify deployment and redundancy.  I'm putting my 
thoughts out on the list, to gather responses, to see if there's a 
better way.

>
>
>   
>> Jini's lookup service lack of AND / OR querying capability is due to
>> security, the avoidance of instantiating foreign objects.
>>
>> Delayed unmarshalling of the service proxy allows service entry's to be
>> compared as objects, without requiring a codebase download for the proxy if
>> it's not the service we want, so it's not quite just returning a
>> MarshalledInstance.  This should be done without compromising the good
>> security features of the existing lookup service.
>>
>>
>>     
> I don't follow - I could tweak ServiceItem to hold the proxy as a
> MarshalledInstance and still expose all other "service identifying
> information". That MarshalledInstance mightn't even immediately carry the
> proxy code, could still be on the server and pulled down at point the
> consumer actually wants the proxy. Feels like some simple
> interface/sub-classing.
>   

This is true for cases where the service types don't matter to the 
client, I think Gregg wanted to elimate all codebase downloads until he 
was sure he had the correct service. Use of a MarshalledInstance could 
be an acceptable compromise, if Entry's have their own codebase 
annotations.  How would this affect the lookup semantics if we're 
looking for particular service types though?

>
>
>   
>> Over the internet, we could potentially have very large lookup services, by
>> allowing clients to remove unwanted services from their results before
>> unmarshalling, we can reduce the resources required of the client:
>>
>>   * Network Bandwidth, clients don't need to download unwanted codebases.
>>   * Memory (ClassLoader and unwanted classes are not loaded into memory).
>>
>>
>>     
> If we were to go across internet have we squared away use of e.g. DNS-SD?
> i.e. Is it a given we'll expose a classic JINI LUS?
>   

The semantics of DNS-SD make it well suited to discovery of a lookup 
service.  I think Sim highlighted earlier that its difficult to get 
domain administrators to do Dynamic Updated DNS-SD, they're comfortable 
with DNS-SRV records, so it would appear easier to rely on DNS-SD as a 
lookup locator / domain browser, not as a lookup service replacement.

Cheers,

Peter.

Re: MarshalledServiceItem

[MICHAEL MCGRADY <mm...@topiatechnology.com>]

+1
On Feb 1, 2011, at 6:30 AM, Patricia Shanahan wrote:

> Sim IJskes - QCG wrote:
>> On 01-02-11 12:40, Dan Creswell wrote:
>>> Ah, I know Sim is gonna hate this but I feel the need to retain full context
>>> for now....
>> What i would prefer to see is:
>> - a use case defining internet deployment
>> - a proof of concept prototype implementing this use case
>> - define lessons learned
>> If it turns out that there is something in the lookup what needs improvement, then we can change that.
>> Better not use internet deployment in requirements discussions before whe have defined what it is. Thats too vague for me.
> 
> I agree.
> 
> If implementing Internet deployment is either impossible regardless of the proposed changes, or can be done well without them, then making the changes would be a mistake.
> 
> The easiest way to be sure about this is a use-case and proof of concept prototype. The prototype can be used to demonstrate how proposed River changes contribute to implementing the use-case.
> 
> Patricia
> 

Michael McGrady
Chief Architect
Topia Technology, Inc.
Cel 1.253.720.3365
Work 1.253.572.9712 extension 2037
mmcgrady@topiatechnology.com

Re: MarshalledServiceItem

[MICHAEL MCGRADY <mm...@topiatechnology.com>]

+1
On Feb 1, 2011, at 6:35 AM, Patricia Shanahan wrote:

> Patricia Shanahan wrote:
>> Sim IJskes - QCG wrote:
>>> On 01-02-11 12:40, Dan Creswell wrote:
>>>> Ah, I know Sim is gonna hate this but I feel the need to retain full context
>>>> for now....
>>> 
>>> What i would prefer to see is:
>>> - a use case defining internet deployment
>>> - a proof of concept prototype implementing this use case
>>> - define lessons learned
>>> 
>>> If it turns out that there is something in the lookup what needs improvement, then we can change that.
>>> 
>>> Better not use internet deployment in requirements discussions before whe have defined what it is. Thats too vague for me.
>> I agree.
>> If implementing Internet deployment is either impossible regardless of the proposed changes, or can be done well without them, then making the changes would be a mistake.
>> The easiest way to be sure about this is a use-case and proof of concept prototype. The prototype can be used to demonstrate how proposed River changes contribute to implementing the use-case.
> 
> Note also that even current proposals are exactly right about what needs to be changed in River for Internet deployment, this sequence does not make any change in the total amount of work, just in the order in which it is done.
> 
> The use-case is the core of a document explaining how to do Internet deployment, and the proof-of-concept prototype is the foundation for the example code we would need.
> 
> Patricia

Michael McGrady
Chief Architect
Topia Technology, Inc.
Cel 1.253.720.3365
Work 1.253.572.9712 extension 2037
mmcgrady@topiatechnology.com

Re: MarshalledServiceItem

[Patricia Shanahan <pa...@acm.org>]

Patricia Shanahan wrote:
> Sim IJskes - QCG wrote:
>> On 01-02-11 12:40, Dan Creswell wrote:
>>> Ah, I know Sim is gonna hate this but I feel the need to retain full 
>>> context
>>> for now....
>>
>> What i would prefer to see is:
>> - a use case defining internet deployment
>> - a proof of concept prototype implementing this use case
>> - define lessons learned
>>
>> If it turns out that there is something in the lookup what needs 
>> improvement, then we can change that.
>>
>> Better not use internet deployment in requirements discussions before 
>> whe have defined what it is. Thats too vague for me.
> 
> I agree.
> 
> If implementing Internet deployment is either impossible regardless of 
> the proposed changes, or can be done well without them, then making the 
> changes would be a mistake.
> 
> The easiest way to be sure about this is a use-case and proof of concept 
> prototype. The prototype can be used to demonstrate how proposed River 
> changes contribute to implementing the use-case.

Note also that even current proposals are exactly right about what needs 
to be changed in River for Internet deployment, this sequence does not 
make any change in the total amount of work, just in the order in which 
it is done.

The use-case is the core of a document explaining how to do Internet 
deployment, and the proof-of-concept prototype is the foundation for the 
example code we would need.

Patricia

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Patricia Shanahan wrote:
> Sim IJskes - QCG wrote:
>> On 01-02-11 12:40, Dan Creswell wrote:
>>> Ah, I know Sim is gonna hate this but I feel the need to retain full 
>>> context
>>> for now....
>>
>> What i would prefer to see is:
>> - a use case defining internet deployment
>> - a proof of concept prototype implementing this use case
>> - define lessons learned
>>
>> If it turns out that there is something in the lookup what needs 
>> improvement, then we can change that.
>>
>> Better not use internet deployment in requirements discussions before 
>> whe have defined what it is. Thats too vague for me.
>
> I agree.
>
> If implementing Internet deployment is either impossible regardless of 
> the proposed changes, or can be done well without them, then making 
> the changes would be a mistake.
>
> The easiest way to be sure about this is a use-case and proof of 
> concept prototype. The prototype can be used to demonstrate how 
> proposed River changes contribute to implementing the use-case.
>
> Patricia
>
>

I think we need a standard process for developing and proposing new 
components.  Gregg provided his reef project in 2006, but I don't think 
anyone from the River / Jini community has given much feedback or been 
interested in changes to lookup semantics.

Jini was targeted at devices, and corporate networks, but Jini services 
are not made available publicly over the internet.  Jini was designed 
with failure in mind and the 8 fallacies of distributed computing.  The 
internet is a form of distributed computing.

Jini works well on trusted networks, but untrusted networks are difficult.

Jini works well on networks with high bandwidth low latency, but has 
usability issues with low bandwidth and high latency connections, or in 
Gregg's case, probably has some issues with high bandwith networks also.

Gregg and Chris have come up with solutions to low bandwith, high 
latency networks, but these are still private networks.

So to create a proof of concept prototype for the internet there's a 
need to address at least the following issues:

    * Low bandwidth high latency connections.
    * Security on untrusted networks.
    * Firewall traversal.
    * Improved service search ability.
    * Discovery of internet provided services.

I apologise in advance for use cases, which may or may not be a correct 
interpretation of what "Use case" means.

A complex use case might be:

One company has a contract with a second company, a supplier which 
provides parts and components the first company uses in manufacture of 
it's products, the two company's have formed a contract based on cost 
plus markup, where the first company uses the buying power of the second 
to take advantage of volume discounts, the companies have agreed on 
standard interfaces for jini services.  The second company engages 
multiple manufacturers of components, these manufacturers provide supply 
chain product information, such as models, fit up drawings, pictures and 
information relating to the product it sells, each manufacturer utilises 
it's own internal systems and use their own Service UI's to display 
information regarding supplied components for downstream purchasers and 
up to date service bulletins for customers maintenance advise systems, 
they also implement standard agreed service interfaces for commerce.

The second company, the supplier, also has other customers, these 
company's use the services to enable their different systems to interact 
using Jini service interfaces and Service UI's, rather than protocols.  
This needs to be done securely and records of all transactions need to 
be retained by all parties.

These services are performed over the internet, rather than through 
virtual private networks, some applications are required in field for 
maintenance and service bulletins, where the network may be third party 
or public.

A simple use case might be:

A user browses services online, that perform various functions and are 
found using a Jini service browser, all Service UI are downloaded on 
demand, applications might range from online sellers and purchasers, to 
interactive information, games or online banking.

Cheers,

Peter.

Re: MarshalledServiceItem

[Patricia Shanahan <pa...@acm.org>]

Sim IJskes - QCG wrote:
> On 01-02-11 12:40, Dan Creswell wrote:
>> Ah, I know Sim is gonna hate this but I feel the need to retain full 
>> context
>> for now....
> 
> What i would prefer to see is:
> - a use case defining internet deployment
> - a proof of concept prototype implementing this use case
> - define lessons learned
> 
> If it turns out that there is something in the lookup what needs 
> improvement, then we can change that.
> 
> Better not use internet deployment in requirements discussions before 
> whe have defined what it is. Thats too vague for me.

I agree.

If implementing Internet deployment is either impossible regardless of 
the proposed changes, or can be done well without them, then making the 
changes would be a mistake.

The easiest way to be sure about this is a use-case and proof of concept 
prototype. The prototype can be used to demonstrate how proposed River 
changes contribute to implementing the use-case.

Patricia

Re: MarshalledServiceItem

[Sim IJskes - QCG <si...@qcg.nl>]

On 01-02-11 12:40, Dan Creswell wrote:
> Ah, I know Sim is gonna hate this but I feel the need to retain full context
> for now....

What i would prefer to see is:
- a use case defining internet deployment
- a proof of concept prototype implementing this use case
- define lessons learned

If it turns out that there is something in the lookup what needs 
improvement, then we can change that.

Better not use internet deployment in requirements discussions before 
whe have defined what it is. Thats too vague for me.

Gr. Sim

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

Ah, I know Sim is gonna hate this but I feel the need to retain full context
for now....

On 1 February 2011 10:55, Peter Firmstone <ji...@zeus.net.au> wrote:

> Dan,
>
> I've appended the updated class after our earlier discussion,
> MarshalledServiceItem, now only has one method as suggested.
>
> An option is to consider giving Entry's their own jar files, and to
> consider doing something about providing a new codebase URL Handler, for
> separating identity from location, so we can discover the same jar files in
> other locations,


I don't think I understand the separation of identity from location fully
yet so let's try a framing statement:

A Service's ID and it's type plus some minimum set of Entrys to identify
pertinent service details is enough isn't it?. And any client wishing to
identify such a service would require those minimal Entry's (agreed it
mightn't want the others, could filter that at Lookup Service side such that
it never gets to the client thus doesn't need downloading).

I don't see the need to play marshalling games so much as delay opening up
the proxy and limiting the Entry's a client sees to those it declares
interest in.


> similar to Codebase Services with message digests or Maven Provisioning as
> suggested by Dennis.  This doesn't mean I don't support the idea that the
> best location for a codebase is at the originating service node, just that
> it's likely that many service proxy's could share identical jar files, so
> why should the client re-download something it already has?
>
>
But, if service proxy's are sharing jar files, what does that mean? Someone
somewhere chose to package them all together like that for some reason. What
reason, what problem are they solving?

Again, I'm sitting here thinking as a deployer of services, if I want to
widget around and consolidate .jars I can do that ahead of time and then
tweak service codebases via config just prior to deploy.



> Jini's lookup service lack of AND / OR querying capability is due to
> security, the avoidance of instantiating foreign objects.
>
> Delayed unmarshalling of the service proxy allows service entry's to be
> compared as objects, without requiring a codebase download for the proxy if
> it's not the service we want, so it's not quite just returning a
> MarshalledInstance.  This should be done without compromising the good
> security features of the existing lookup service.
>
>
I don't follow - I could tweak ServiceItem to hold the proxy as a
MarshalledInstance and still expose all other "service identifying
information". That MarshalledInstance mightn't even immediately carry the
proxy code, could still be on the server and pulled down at point the
consumer actually wants the proxy. Feels like some simple
interface/sub-classing.



> Over the internet, we could potentially have very large lookup services, by
> allowing clients to remove unwanted services from their results before
> unmarshalling, we can reduce the resources required of the client:
>
>   * Network Bandwidth, clients don't need to download unwanted codebases.
>   * Memory (ClassLoader and unwanted classes are not loaded into memory).
>
>
If we were to go across internet have we squared away use of e.g. DNS-SD?
i.e. Is it a given we'll expose a classic JINI LUS?


> The intent behind MarshalledServiceItem, is the entry's available for
> comparison are only those required by the client, these can be loaded into a
> child classloader (if they don't exist on the classpath) and shared by all
> MarshalledServiceItem instances.  You wouldn't require the Entry for a
> ServiceUI without it's service proxy for instance.
>
>
I think I get the intent but I'm far from convinced that all these
marshalling games and such are necessary (see above framing comment).


> After the MarshalledServiceItem has been unmarshalled, the Entry's can be
> from the classpath and the proxy's classloader (meaning we might need to
> unmarshall some Entry's a second time).
>
> I don't doubt the requirement for more efficient and flexible means of
> service lookup, just the interfaces so far proposed, which is why I'm giving
> it lots of time & hoping for plenty of review, before setting it in stone.
>
> The reality is the line between private networks and the internet is
> blurring, in spite of router NAT.  The internet is the market most ready to
> try different ideas like Jini / River, the corporate intranet is the least
> likely place and for political reasons more than technical ones.  I'd like
> to take River where it hasn't been, where there are no users at present to
> justify such a feature.
>
> Yeah I know there are other issues that need solving, like firewall
> traversal.
>
> P.S. Hope you're feeling better after the coffee ;)
>
>
Ops mess hasn't cleared up so alas, no :)


>
> Cheers,
>
>
Thanks for tolerating my rantiness!


> Peter.
>
> Dan Creswell wrote:
>
>> So...
>>
>> On 1 February 2011 01:29, Gregg Wonderly <gr...@wonderly.org> wrote:
>>
>>
>>
>>> One of the important things for my use of my changes to the
>>> ServiceRegistrar interface was separating deserialization of the Entrys
>>> from
>>> the service object.  Ultimately, I wanted to not have to worry about how
>>> the
>>> "codebase" was structured in order to minimize downloading. Some Jini
>>> deployers have done things to put a small jar at the front of the
>>> codebase
>>> that just has the preferred.list in it.  Even that download, for me,
>>> would
>>> have been too much.  My customers' deployments have nearly 50 codebases
>>> visible when my client starts up.  Those 50 connections over a cellular
>>> or
>>> other high latency network would make service selection take way too
>>> long.
>>>
>>>
>>>
>>>
>> "deserialization of Entry's from the service object"? What does "service
>> object" refer to?
>>
>> I think you _have_ to worry about how the codebase is structured. Anything
>> else is a hack and if we're going to do hacks there are better ones like
>> bundling up people's .jars for them etc so save on connections/add caching
>> and such. Doing it further downstream in the lookup mechanism is treating
>> a
>> symptom not the problem. Not the sort of thing that keeps a design clean
>> or
>> consistent.
>>
>>
>>
>> Being able to defer downloading was also controlled by the changes to
>>
>>
>>> ClassLoading to include the "neverPrefer" settings on a class name.  I
>>> can
>>> do that now through the recent RMIClassLoaderSPI override capabilities.
>>>  But
>>> I still need to be able to control which Entry is deserialized.  I need
>>> to
>>> be able to ask what classes, by name, are in the namespace of each Entry
>>> object as my implementation allowed. I had made changes to Reggie
>>> marshalling to use reflection to get the class hierarchy, and then
>>> provided
>>> access to the list of classnames as part of the returned, marshalled
>>> value
>>> in my API.
>>>
>>>
>>>
>>>
>> Digging through classes belonging to a service implementation that is
>> designed to be encapsulated/not your concern? Does that sound right? I
>> gotta
>> say, it sounds horribly invasive, very hacky and speaks of an overly
>> complex
>> solution as the result of treating symptoms, not problems.
>>
>> Which brings me to a point Peter made elsewhere: "delayed umarshalling and
>> selective unmarshalling of Entry's are very important features, missing
>> from
>> River."
>>
>> Important to whom? How many people are in the category?
>>
>> If it's a lot of people, we should spend some time evaluating whether the
>> default class resolution/download/lookup approach is the generic norm and
>> potential replacements.
>>
>> Simplistically, delaying downloading is as simple as returning a
>> MarshalledInstance on which one can then call .get as and when one
>> desires.....
>>
>> Now, I haven't had my coffee yet and I've got a pile of operational
>> worries
>> to attend to this morning which means I'm maybe a little shout'y,
>> nevertheless I for one am not convinced of the value of this and less
>> convinced we have the right solution which feels complex, exceptional and
>> clunky.
>>
>>
>>
>>
> /*
> * Licensed to the Apache Software Foundation (ASF) under one
> * or more contributor license agreements.  See the NOTICE file
> * distributed with this work for additional information
> * regarding copyright ownership. The ASF licenses this file
> * to you under the Apache License, Version 2.0 (the
> * "License"); you may not use this file except in compliance
> * with the License. You may obtain a copy of the License at
> *
> *      http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
>
> package org.apache.river.api.lookup;
>
> import java.net.URI;
> import java.security.CodeSource;
> import net.jini.core.entry.Entry;
> import net.jini.core.lookup.ServiceID;
> import net.jini.core.lookup.ServiceItem;
>
> /**
> * MarshalledServiceItem extends ServiceItem and can be used anywhere a
> * ServiceItem can.  A MarshalledServiceItem implementation instance
> * contains the marshalled form of a Service and it's Entry's,
> * the corresponding superclass ServiceItem however contains null values
> * for the service and can exclude any Entry's, however where Entry
> * classes already exist at the client, that they be unmarshalled.
> *
> * The ServiceID shall be in unmarshalled form always in the ServiceItem
> super class.
> *
> * Since the ServiceItem.service is null, use of this class in existing
> software
> * will not return the service, however it will not break that software as
> * ServiceItem's contract is to set service or Entry's to null when they
> cannot
> * be unmarshalled.
> *
> * ServiceItem's toString() method will return a different result for
> * MarshalledServiceItem instances.
> *
> * If required, a new ServiceItem that is fully unmarshalled
> * can be constructed from this class's methods and ServiceID.
> *
> * @author Peter Firmstone.
> */
> public abstract class MarshalledServiceItem extends ServiceItem{
>   private static final long SerialVersionUID = 1L;
>   protected MarshalledServiceItem(ServiceID id, Entry[]
> unmarshalledEntries){
>       super(id, (Object) null, unmarshalledEntries);
>   }
>   /**
>    * Unmarshall the ServiceItem.  The returned ServiceItem may have a null
> service
>    * or entries , if unmarshalling is unsuccessful.
>    *
>    * @return ServiceItem, totally unmarshalled.
>    */
>   public abstract ServiceItem unmarshall();
> }
>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Dan,

I've appended the updated class after our earlier discussion, 
MarshalledServiceItem, now only has one method as suggested.

An option is to consider giving Entry's their own jar files, and to 
consider doing something about providing a new codebase URL Handler, for 
separating identity from location, so we can discover the same jar files 
in other locations, similar to Codebase Services with message digests or 
Maven Provisioning as suggested by Dennis.  This doesn't mean I don't 
support the idea that the best location for a codebase is at the 
originating service node, just that it's likely that many service 
proxy's could share identical jar files, so why should the client 
re-download something it already has?

Jini's lookup service lack of AND / OR querying capability is due to 
security, the avoidance of instantiating foreign objects.

Delayed unmarshalling of the service proxy allows service entry's to be 
compared as objects, without requiring a codebase download for the proxy 
if it's not the service we want, so it's not quite just returning a 
MarshalledInstance.  This should be done without compromising the good 
security features of the existing lookup service.

Over the internet, we could potentially have very large lookup services, 
by allowing clients to remove unwanted services from their results 
before unmarshalling, we can reduce the resources required of the client:

    * Network Bandwidth, clients don't need to download unwanted codebases.
    * Memory (ClassLoader and unwanted classes are not loaded into memory).

The intent behind MarshalledServiceItem, is the entry's available for 
comparison are only those required by the client, these can be loaded 
into a child classloader (if they don't exist on the classpath) and 
shared by all MarshalledServiceItem instances.  You wouldn't require the 
Entry for a ServiceUI without it's service proxy for instance.

After the MarshalledServiceItem has been unmarshalled, the Entry's can 
be from the classpath and the proxy's classloader (meaning we might need 
to unmarshall some Entry's a second time).
 
I don't doubt the requirement for more efficient and flexible means of 
service lookup, just the interfaces so far proposed, which is why I'm 
giving it lots of time & hoping for plenty of review, before setting it 
in stone.

The reality is the line between private networks and the internet is 
blurring, in spite of router NAT.  The internet is the market most ready 
to try different ideas like Jini / River, the corporate intranet is the 
least likely place and for political reasons more than technical ones.  
I'd like to take River where it hasn't been, where there are no users at 
present to justify such a feature.

Yeah I know there are other issues that need solving, like firewall 
traversal.

P.S. Hope you're feeling better after the coffee ;)

Cheers,

Peter.

Dan Creswell wrote:
> So...
>
> On 1 February 2011 01:29, Gregg Wonderly <gr...@wonderly.org> wrote:
>
>   
>> One of the important things for my use of my changes to the
>> ServiceRegistrar interface was separating deserialization of the Entrys from
>> the service object.  Ultimately, I wanted to not have to worry about how the
>> "codebase" was structured in order to minimize downloading. Some Jini
>> deployers have done things to put a small jar at the front of the codebase
>> that just has the preferred.list in it.  Even that download, for me, would
>> have been too much.  My customers' deployments have nearly 50 codebases
>> visible when my client starts up.  Those 50 connections over a cellular or
>> other high latency network would make service selection take way too long.
>>
>>
>>     
> "deserialization of Entry's from the service object"? What does "service
> object" refer to?
>
> I think you _have_ to worry about how the codebase is structured. Anything
> else is a hack and if we're going to do hacks there are better ones like
> bundling up people's .jars for them etc so save on connections/add caching
> and such. Doing it further downstream in the lookup mechanism is treating a
> symptom not the problem. Not the sort of thing that keeps a design clean or
> consistent.
>
>
>
> Being able to defer downloading was also controlled by the changes to
>   
>> ClassLoading to include the "neverPrefer" settings on a class name.  I can
>> do that now through the recent RMIClassLoaderSPI override capabilities.  But
>> I still need to be able to control which Entry is deserialized.  I need to
>> be able to ask what classes, by name, are in the namespace of each Entry
>> object as my implementation allowed. I had made changes to Reggie
>> marshalling to use reflection to get the class hierarchy, and then provided
>> access to the list of classnames as part of the returned, marshalled value
>> in my API.
>>
>>
>>     
> Digging through classes belonging to a service implementation that is
> designed to be encapsulated/not your concern? Does that sound right? I gotta
> say, it sounds horribly invasive, very hacky and speaks of an overly complex
> solution as the result of treating symptoms, not problems.
>
> Which brings me to a point Peter made elsewhere: "delayed umarshalling and
> selective unmarshalling of Entry's are very important features, missing from
> River."
>
> Important to whom? How many people are in the category?
>
> If it's a lot of people, we should spend some time evaluating whether the
> default class resolution/download/lookup approach is the generic norm and
> potential replacements.
>
> Simplistically, delaying downloading is as simple as returning a
> MarshalledInstance on which one can then call .get as and when one
> desires.....
>
> Now, I haven't had my coffee yet and I've got a pile of operational worries
> to attend to this morning which means I'm maybe a little shout'y,
> nevertheless I for one am not convinced of the value of this and less
> convinced we have the right solution which feels complex, exceptional and
> clunky.
>
>
>   
/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.river.api.lookup;

import java.net.URI;
import java.security.CodeSource;
import net.jini.core.entry.Entry;
import net.jini.core.lookup.ServiceID;
import net.jini.core.lookup.ServiceItem;

/**
 * MarshalledServiceItem extends ServiceItem and can be used anywhere a
 * ServiceItem can.  A MarshalledServiceItem implementation instance
 * contains the marshalled form of a Service and it's Entry's,
 * the corresponding superclass ServiceItem however contains null values
 * for the service and can exclude any Entry's, however where Entry
 * classes already exist at the client, that they be unmarshalled.
 *
 * The ServiceID shall be in unmarshalled form always in the ServiceItem 
super class.
 *
 * Since the ServiceItem.service is null, use of this class in existing 
software
 * will not return the service, however it will not break that software as
 * ServiceItem's contract is to set service or Entry's to null when they 
cannot
 * be unmarshalled.
 *
 * ServiceItem's toString() method will return a different result for
 * MarshalledServiceItem instances.
 *
 * If required, a new ServiceItem that is fully unmarshalled
 * can be constructed from this class's methods and ServiceID.
 *
 * @author Peter Firmstone.
 */
public abstract class MarshalledServiceItem extends ServiceItem{
    private static final long SerialVersionUID = 1L;
    protected MarshalledServiceItem(ServiceID id, Entry[] 
unmarshalledEntries){
        super(id, (Object) null, unmarshalledEntries);
    }
    /**
     * Unmarshall the ServiceItem.  The returned ServiceItem may have a 
null service
     * or entries , if unmarshalling is unsuccessful.
     *
     * @return ServiceItem, totally unmarshalled.
     */
    public abstract ServiceItem unmarshall();
}

Re: MarshalledServiceItem

[Michał Kłeczek <mi...@xpro.biz>]

Regardless of whether LUS should be used for service browsing or
another service should be used - I can see the issue with
PREFERRED.LIST. It is really unfortunate that the client must download
something just to make a decision whether it must download anything...
Thinking out loud - isn't it (yet another) case for annotating classes
with objects - as I proposed in another thread discussing possible
unmarshalling DoS attacks. That way we could provide the client with
some class loading information (namely - preferred list but possibly
something else as well) in band while still maintaining out of band
code download.

Michal

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

ClasspathRestrictedServiceItem?

Dan Creswell wrote:
> Thing is the ServiceItem is not the thing being resolved....
>
> On 3 February 2011 09:59, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> How about LocallyResolvedServiceItem?
>>
>>
>> Dan Creswell wrote:
>>
>>     
>>> Sure, the method does indeed unmarshall but within specific constraints
>>> and
>>> those constraints are defined by the enclosing class (an instance of
>>> ServiceItem in this case).
>>>
>>> So I'm thinking we're really talking about is a ServiceItem that pulls
>>> from
>>> classpath so maybe LocallyResolvingServiceItem (not entirely keen on that
>>> but explains what's happening).
>>>
>>> On some level there's a bit of me that screams we have a broken design if
>>> we
>>> can't get decent names.....
>>>
>>> On 3 February 2011 09:10, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>
>>>
>>>
>>>       
>>>> Dan Creswell wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> "In summary MarshalledServiceItem is a class that only resolves local
>>>>> code.
>>>>>  Interestingly MarshalledServiceItem, might not be an appropriate name,
>>>>> perhaps LocalServiceItem to indicate that only local code is utilised,
>>>>> on
>>>>> this reasoning, I can't see any objection to reflective proxy's being
>>>>> non
>>>>> null either."
>>>>>
>>>>> Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
>>>>> what's going on.....
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> The method unmarshall, sort of makes sense, can you think of a better
>>>> name?
>>>>
>>>>
>>>>
>>>>         
>>>
>>>       
>>     
>
>   

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

We're not matching, we're filtering to ignore attributes we have no interest
in (which in turn avoids unnecessary codebase access). So we're simply
indicating the types of things we are prepared to accept thus:

ServiceInfo.class, Name.class

As opposed to:

new ServiceInfo(....), new Name(.....)

Matching is done as a standard part of lookup and is tuplespace'ish already.

On 5 February 2011 19:41, <jg...@simulexinc.com> wrote:

> To state the obvious, matching entries that the client passes in sounds
> like tuplespace template matching.
>
> Perhaps this is a case where we'd benefit from having around a tuplespace
> data structure, in addition to the service?
>
> jamesG
>
> -----Original Message-----
> From: "Peter Firmstone" <ji...@zeus.net.au>
> Sent: Saturday, February 5, 2011 5:28am
> To: river-dev@incubator.apache.org
> Subject: Re: MarshalledServiceItem
>
> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>
> So we make the Entry's the client's interested in, the only Entry's
> that'll be unmarshalled, those and the ServiceID.
>
> Cheers,
>
> Peter.
>
>
>

Re: MarshalledServiceItem

[jg...@simulexinc.com]

To state the obvious, matching entries that the client passes in sounds like tuplespace template matching.   

Perhaps this is a case where we'd benefit from having around a tuplespace data structure, in addition to the service?

jamesG

-----Original Message-----
From: "Peter Firmstone" <ji...@zeus.net.au>
Sent: Saturday, February 5, 2011 5:28am
To: river-dev@incubator.apache.org
Subject: Re: MarshalledServiceItem

Ok, good reasoning, we can keep the Entry's explicit, in the call.

So we make the Entry's the client's interested in, the only Entry's 
that'll be unmarshalled, those and the ServiceID.

Cheers,

Peter.

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Ok, done.

Dan Creswell wrote:
> Solid - probably ought to be a constructor that doesn't add defaults
> too.....and maybe a note about thread safety on add.
>
> On 7 February 2011 04:29, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> Dan Creswell wrote:
>>
>>     
>>>  Sorry, what you meant was the builder generates a default array or Set of
>>>       
>>>> Entry classes required to support the platform, additional Entry classes
>>>> the
>>>> client needs can be added, then it builds the array to pass as a
>>>> parameter
>>>> to the lookup service.
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> Yep.
>>>
>>>
>>>
>>>
>>>       
>> How's this?
>>
>> Cheers,
>>
>> Peter.
>>
>>
>> /*
>> * Licensed to the Apache Software Foundation (ASF) under one
>> * or more contributor license agreements.  See the NOTICE file
>> * distributed with this work for additional information
>> * regarding copyright ownership. The ASF licenses this file
>> * to you under the Apache License, Version 2.0 (the
>> * "License"); you may not use this file except in compliance
>> * with the License. You may obtain a copy of the License at
>> *
>> *      http://www.apache.org/licenses/LICENSE-2.0
>> *
>> * Unless required by applicable law or agreed to in writing, software
>> * distributed under the License is distributed on an "AS IS" BASIS,
>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> * See the License for the specific language governing permissions and
>> * limitations under the License.
>> */
>>
>> package org.apache.river.api.lookup;
>>
>> import java.util.HashSet;
>> import java.util.Set;
>> import net.jini.lookup.entry.Address;
>> import net.jini.lookup.entry.Comment;
>> import net.jini.lookup.entry.Location;
>> import net.jini.lookup.entry.Name;
>> import net.jini.lookup.entry.ServiceInfo;
>> import net.jini.lookup.entry.Status;
>> import net.jini.lookup.entry.UIDescriptor;
>>
>> /**
>> * A little builder utility class that creates an array of Entry classes to
>> * be used as a parameter for StreamServiceRegistrar.  All the platform
>> * Entry's are included.
>> *
>> * Suggested by Dan Creswell.
>> * @author peter
>> */
>> public class DefaultEntries {
>>   private final Set<Class> entrys;
>>   public DefaultEntries() {
>>       entrys = new HashSet<Class>(16);
>>       add(Comment.class);
>>       add(Location.class);
>>       add(Name.class);
>>       add(ServiceInfo.class);
>>       add(Status.class);
>>       add(UIDescriptor.class);
>>       add(Address.class);
>>   }
>>   public DefaultEntries add(Class cl){
>>       entrys.add(cl);
>>       return this;
>>   }
>>   public Class[] getEntries(){
>>       return entrys.toArray(new Class[entrys.size()]);
>>   }
>>
>> }
>>
>>
>>     
>
>   

Re: MarshalledServiceItem

[Sim IJskes - QCG <si...@qcg.nl>]

On 07-02-11 11:56, Peter Firmstone wrote:
> @Override
> public String toString(){
> String newline = System.getProperty("line.separator");
> StringBuilder sb = new StringBuilder(256);
> sb.append("DefaultEntries:");
> sb.append(newline);
> Iterator<Class> it = entrys.iterator();
> while (it.hasNext()){
> sb.append(it.next().getName());
> sb.append(newline);
> }
> return sb.toString();
> }

Can we have the toString fitting in 1 line, ex [a,b,c] prefixed with 
Class.getSimpleName()?

Gr. Sim

Re: MarshalledServiceItem

[Peter <ji...@zeus.net.au>]

----- Original message -----
> Gregg Wonderly wrote:
> > On 2/8/2011 10:51 AM, Patricia Shanahan wrote:
> > > There are some negatives to using CHM compared to wrapping a HashMap in a
> > > synchronized map. It is easy to write methods for compound operations
> > > such as
> > > removing a key-value pair if, and only if, some other key is present.
> >
> > Yes, this is true.  If we do need some compound operations, then it
> > would be necessary to add a lock to synchronize on, and then everything
> > else referencing the structure would need to be synchronized since CHM
> > uses internal locking which does not involve synchronized(this).
>
> Or switch implementations if there is a problem with the initial choice.
> If the code uses the Map interface, implemented by both CHM and a
> synchronized map wrapping a HashMap, it should just be a matter of
> changing the initialization expression.
>
> Patricia

Well almost, put if absent requires a check on the return value, often you need to create a new object, then do a put if absent, if there was an intervening put, throw away the new object and instead use the existing one.

The synchronised collections wrappers are directly substitutable, I've created similar multi read single write wrappers that are directly substitutable, these also return a snapshot iterator that won't throw a concurrent modification exception.

The good part about using the wrappers is that concurrency concerns are not reimplemented in each class, instead you've only got one place to do concurrency debugging and maintenance.

Peter.

Re: MarshalledServiceItem

[Patricia Shanahan <pa...@acm.org>]

Gregg Wonderly wrote:
> On 2/8/2011 10:51 AM, Patricia Shanahan wrote:
>> There are some negatives to using CHM compared to wrapping a HashMap in a
>> synchronized map. It is easy to write methods for compound operations 
>> such as
>> removing a key-value pair if, and only if, some other key is present.
> 
> Yes, this is true.  If we do need some compound operations, then it 
> would be necessary to add a lock to synchronize on, and then everything 
> else referencing the structure would need to be synchronized since CHM 
> uses internal locking which does not involve synchronized(this).

Or switch implementations if there is a problem with the initial choice. 
If the code uses the Map interface, implemented by both CHM and a 
synchronized map wrapping a HashMap, it should just be a matter of 
changing the initialization expression.

Patricia

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

On 2/8/2011 10:51 AM, Patricia Shanahan wrote:
> There are some negatives to using CHM compared to wrapping a HashMap in a
> synchronized map. It is easy to write methods for compound operations such as
> removing a key-value pair if, and only if, some other key is present.

Yes, this is true.  If we do need some compound operations, then it would be 
necessary to add a lock to synchronize on, and then everything else referencing 
the structure would need to be synchronized since CHM uses internal locking 
which does not involve synchronized(this).

Gregg Wonderly

>
> On 2/8/2011 8:26 AM, Dan Creswell wrote:
>> No friction, simplest thing that could possibly work is the only thing
>> running through my head.
>>
>> I'm happy to see CHM used if people deem it appropriate - as is so often the
>> case, I'm not writing the code so my opinion should only carry so much
>> weight.
>>
>> On 8 February 2011 15:12, Gregg Wonderly<gr...@wonderly.org> wrote:
>>
>>> On 2/8/2011 9:00 AM, Dan Creswell wrote:
>>>
>>>> It's all about expected usage patterns.
>>>>
>>>> Locking in this case will be per instance and the likelihood of contention
>>>> is low given no one's going to call add a lot or indeed toString a lot in
>>>> a
>>>> short space of time.
>>>>
>>>> So given:
>>>>
>>>> Define how concurrency is handled then implement
>>>>
>>>> And assuming it's in line with the above contention model I'd say
>>>> synchronized would be adequate.
>>>>
>>>
>>> I believe this is the case. But, I also know that it can come back to
>>> bite.
>>>
>>> CHM solves it without synchronized, and that, I believe is a better choice
>>> because it eliminates the problem. Any additional methods or usage patterns
>>> that fall out of the life cycle of this object will not have any new or
>>> different behavior or synchronization requirements.
>>>
>>> Is there an attribute of CHM that you find unattractive Dan? I'm not sure
>>> I understand the friction I am sensing.
>>>
>>> Gregg
>>>
>>>
>>> On 8 February 2011 14:54, Gregg Wonderly<gr...@wonderly.org> wrote:
>>>>
>>>> On 2/7/2011 5:28 PM, Peter Firmstone wrote:
>>>>>
>>>>> Do we really need a concurrent version of this class? How often will
>>>>>> Entry
>>>>>> classes change?
>>>>>>
>>>>>>
>>>>> My choice is always to consider concurrency first, unless there is some
>>>>> element of performance that this impacts. The simple fact is that there
>>>>> is
>>>>> a toString() that traverses the contents. So, you either must
>>>>> synchronize
>>>>> all access to the contents, or use a CHM which solves the problem without
>>>>> any future concurrency surprises.
>>>>>
>>>>> I'm really serious about this, because of how I've encountered tons of
>>>>> "synchronized" blocks in river which introduce huge latency in
>>>>> "processing"
>>>>> because they cause everyone to wait in line for stuff which very often is
>>>>> quite independent.
>>>>>
>>>>> Yes, this structure is mostly one time mutated. But, its design exposes
>>>>> a
>>>>> continuous mutation capability. Thus, any "read" of the contents has to
>>>>> take that into account.
>>>>>
>>>>> Gregg Wonderly
>>>>>
>>>>>
>>>>> Gregg Wonderly wrote:
>>>>>
>>>>>>
>>>>>> Yes definitely the case. CHM just makes it a lot easier to not worry
>>>>>>> about map
>>>>>>> and set data structures in terms of concurrent access of mutating data.
>>>>>>> Set
>>>>>>> and HashSet are useful, but not always in concurrent mutating data
>>>>>>> management.
>>>>>>>
>>>>>>> I still prefer Vector over ArrayList just because it eliminates
>>>>>>> iteration
>>>>>>> problems that alway require the same lines of code to resolve.
>>>>>>>
>>>>>>> Gregg
>>>>>>>
>>>>>>> Sent from my iPhone
>>>>>>>
>>>>>>> On Feb 7, 2011, at 2:08 PM, Dan Creswell<da...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> All true, still need to document the concurrent contract for this
>>>>>>> class
>>>>>>>
>>>>>>>> and
>>>>>>>> implement accordingly.
>>>>>>>>
>>>>>>>> i.e. We can say "toString" is thread safe etc and then use
>>>>>>>> ConcurrentHashMap
>>>>>>>> and friends.
>>>>>>>>
>>>>>>>> But we gotta do it right....
>>>>>>>>
>>>>>>>> On 7 February 2011 19:57, Gregg Wonderly<gr...@wonderly.org> wrote:
>>>>>>>>
>>>>>>>> On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sorry, I've got my nit-picking hat on...
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The null check in the equals method isn't needed because you get a
>>>>>>>>>> free one by using "instanceof".
>>>>>>>>>>
>>>>>>>>>> I know you've warned that the class is not thread-safe, but people
>>>>>>>>>> often forget that doing a sysout of an object calls toString(). I'd
>>>>>>>>>> still be tempted to whack some protection into toString to stop it
>>>>>>>>>> blowing up when people start printing lots of debug messages.
>>>>>>>>>>
>>>>>>>>>> Other than that it looks good to me.
>>>>>>>>>>
>>>>>>>>>> I'd suggest that it would be easy enough to just use
>>>>>>>>>>
>>>>>>>>> ConcurrentHashMap
>>>>>>>>> instead of HashSet if we are going to move to Java 5 anyway. Then,
>>>>>>>>> you
>>>>>>>>> don't have to worry about concurrency issues with toString() etc.
>>>>>>>>>
>>>>>>>>> Gregg Wonderly
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Dan Creswell wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Solid - probably ought to be a constructor that doesn't add
>>>>>>>>>>> defaults
>>>>>>>>>>>
>>>>>>>>>>>> too.....and maybe a note about thread safety on add.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The finished article.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> /*
>>>>>>>>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>>>>>>>>> * or more contributor license agreements. See the NOTICE file
>>>>>>>>>>> * distributed with this work for additional information
>>>>>>>>>>> * regarding copyright ownership. The ASF licenses this file
>>>>>>>>>>> * to you under the Apache License, Version 2.0 (the
>>>>>>>>>>> * "License"); you may not use this file except in compliance
>>>>>>>>>>> * with the License. You may obtain a copy of the License at
>>>>>>>>>>> *
>>>>>>>>>>> * http://www.apache.org/licenses/LICENSE-2.0
>>>>>>>>>>> *
>>>>>>>>>>> * Unless required by applicable law or agreed to in writing,
>>>>>>>>>>> software
>>>>>>>>>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>>>>>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>>>>>>>> implied.
>>>>>>>>>>> * See the License for the specific language governing permissions
>>>>>>>>>>> and
>>>>>>>>>>> * limitations under the License.
>>>>>>>>>>> */
>>>>>>>>>>>
>>>>>>>>>>> package org.apache.river.api.lookup;
>>>>>>>>>>>
>>>>>>>>>>> import java.util.HashSet;
>>>>>>>>>>> import java.util.Iterator;
>>>>>>>>>>> import java.util.Set;
>>>>>>>>>>> import net.jini.lookup.entry.Address;
>>>>>>>>>>> import net.jini.lookup.entry.Comment;
>>>>>>>>>>> import net.jini.lookup.entry.Location;
>>>>>>>>>>> import net.jini.lookup.entry.Name;
>>>>>>>>>>> import net.jini.lookup.entry.ServiceInfo;
>>>>>>>>>>> import net.jini.lookup.entry.Status;
>>>>>>>>>>> import net.jini.lookup.entry.UIDescriptor;
>>>>>>>>>>>
>>>>>>>>>>> /**
>>>>>>>>>>> * A little builder utility class that creates an array of Entry
>>>>>>>>>>> classes
>>>>>>>>>>> to
>>>>>>>>>>> * be used as a parameter for StreamServiceRegistrar. All the jini
>>>>>>>>>>> platform
>>>>>>>>>>> * Entry's are included by default.
>>>>>>>>>>> *
>>>>>>>>>>> * Note: This class is not threadsafe, use external synchronization
>>>>>>>>>>> if
>>>>>>>>>>> required.
>>>>>>>>>>> *
>>>>>>>>>>> * Suggested by Dan Creswell.
>>>>>>>>>>> * @author peter
>>>>>>>>>>> */
>>>>>>>>>>> public class DefaultEntries {
>>>>>>>>>>> private final Set<Class> entrys;
>>>>>>>>>>> public DefaultEntries() {
>>>>>>>>>>> entrys = new HashSet<Class>(16);
>>>>>>>>>>> }
>>>>>>>>>>> /**
>>>>>>>>>>> * Add an Entry class.
>>>>>>>>>>> * @param cl - class
>>>>>>>>>>> * @return this
>>>>>>>>>>> */
>>>>>>>>>>> public DefaultEntries add(Class cl){
>>>>>>>>>>> entrys.add(cl);
>>>>>>>>>>> return this;
>>>>>>>>>>> }
>>>>>>>>>>> /**
>>>>>>>>>>> * All all the Jini Platform Entry's
>>>>>>>>>>> * @return
>>>>>>>>>>> */
>>>>>>>>>>> public DefaultEntries addPlatformEntries(){
>>>>>>>>>>> add(Comment.class);
>>>>>>>>>>> add(Location.class);
>>>>>>>>>>> add(Name.class);
>>>>>>>>>>> add(ServiceInfo.class);
>>>>>>>>>>> add(Status.class);
>>>>>>>>>>> add(UIDescriptor.class);
>>>>>>>>>>> add(Address.class);
>>>>>>>>>>> return this;
>>>>>>>>>>> }
>>>>>>>>>>> /**
>>>>>>>>>>> * Remove all Entry's
>>>>>>>>>>> */
>>>>>>>>>>> public void reset(){
>>>>>>>>>>> entrys.clear();
>>>>>>>>>>> }
>>>>>>>>>>> /**
>>>>>>>>>>> * Generate a new array containing all Entry's added since last
>>>>>>>>>>> reset.
>>>>>>>>>>> * @return
>>>>>>>>>>> */
>>>>>>>>>>> public Class[] getEntries(){
>>>>>>>>>>> return entrys.toArray(new Class[entrys.size()]);
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> @Override
>>>>>>>>>>> public int hashCode() {
>>>>>>>>>>> int hash = 3;
>>>>>>>>>>> hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>>>>>>>>>> 0);
>>>>>>>>>>> return hash;
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> @Override
>>>>>>>>>>> public boolean equals(Object o){
>>>>>>>>>>> if (o == null) return false;
>>>>>>>>>>> if (o instanceof DefaultEntries){
>>>>>>>>>>> if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>>>>>>>>> }
>>>>>>>>>>> return false;
>>>>>>>>>>> }
>>>>>>>>>>> @Override
>>>>>>>>>>> public String toString(){
>>>>>>>>>>> String newline = System.getProperty("line.separator");
>>>>>>>>>>> StringBuilder sb = new StringBuilder(256);
>>>>>>>>>>> sb.append("DefaultEntries:");
>>>>>>>>>>> sb.append(newline);
>>>>>>>>>>> Iterator<Class> it = entrys.iterator();
>>>>>>>>>>> while (it.hasNext()){
>>>>>>>>>>> sb.append(it.next().getName());
>>>>>>>>>>> sb.append(newline);
>>>>>>>>>>> }
>>>>>>>>>>> return sb.toString();
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>

Re: MarshalledServiceItem

[Patricia Shanahan <pa...@acm.org>]

There are some negatives to using CHM compared to wrapping a HashMap in 
a synchronized map. It is easy to write methods for compound operations 
such as removing a key-value pair if, and only if, some other key is 
present.

Patricia


On 2/8/2011 8:26 AM, Dan Creswell wrote:
> No friction, simplest thing that could possibly work is the only thing
> running through my head.
>
> I'm happy to see CHM used if people deem it appropriate - as is so often the
> case, I'm not writing the code so my opinion should only carry so much
> weight.
>
> On 8 February 2011 15:12, Gregg Wonderly<gr...@wonderly.org>  wrote:
>
>> On 2/8/2011 9:00 AM, Dan Creswell wrote:
>>
>>> It's all about expected usage patterns.
>>>
>>> Locking in this case will be per instance and the likelihood of contention
>>> is low given no one's going to call add a lot or indeed toString a lot in
>>> a
>>> short space of time.
>>>
>>> So given:
>>>
>>> Define how concurrency is handled then implement
>>>
>>> And assuming it's in line with the above contention model I'd say
>>> synchronized would be adequate.
>>>
>>
>> I believe this is the case.  But, I also know that it can come back to
>> bite.
>>
>> CHM solves it without synchronized, and that, I believe is a better choice
>> because it eliminates the problem.  Any additional methods or usage patterns
>> that fall out of the life cycle of this object will not have any new or
>> different behavior or synchronization requirements.
>>
>> Is there an attribute of CHM that you find unattractive Dan?  I'm not sure
>> I understand the friction I am sensing.
>>
>> Gregg
>>
>>
>>   On 8 February 2011 14:54, Gregg Wonderly<gr...@wonderly.org>   wrote:
>>>
>>>   On 2/7/2011 5:28 PM, Peter Firmstone wrote:
>>>>
>>>>   Do we really need a concurrent version of this class? How often will
>>>>> Entry
>>>>> classes change?
>>>>>
>>>>>
>>>> My choice is always to consider concurrency first, unless there is some
>>>> element of performance that this impacts.  The simple fact is that there
>>>> is
>>>> a toString() that traverses the contents.  So, you either must
>>>> synchronize
>>>> all access to the contents, or use a CHM which solves the problem without
>>>> any future concurrency surprises.
>>>>
>>>> I'm really serious about this, because of how I've encountered tons of
>>>> "synchronized" blocks in river which introduce huge latency in
>>>> "processing"
>>>> because they cause everyone to wait in line for stuff which very often is
>>>> quite independent.
>>>>
>>>> Yes, this structure is mostly one time mutated.  But, its design exposes
>>>> a
>>>> continuous mutation capability.  Thus, any "read" of the contents has to
>>>> take that into account.
>>>>
>>>> Gregg Wonderly
>>>>
>>>>
>>>>   Gregg Wonderly wrote:
>>>>
>>>>>
>>>>>   Yes definitely the case. CHM just makes it a lot easier to not worry
>>>>>> about map
>>>>>> and set data structures in terms of concurrent access of mutating data.
>>>>>> Set
>>>>>> and HashSet are useful, but not always in concurrent mutating data
>>>>>> management.
>>>>>>
>>>>>> I still prefer Vector over ArrayList just because it eliminates
>>>>>> iteration
>>>>>> problems that alway require the same lines of code to resolve.
>>>>>>
>>>>>> Gregg
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Feb 7, 2011, at 2:08 PM, Dan Creswell<da...@gmail.com>
>>>>>>   wrote:
>>>>>>
>>>>>>   All true, still need to document the concurrent contract for this
>>>>>> class
>>>>>>
>>>>>>> and
>>>>>>> implement accordingly.
>>>>>>>
>>>>>>> i.e. We can say "toString" is thread safe etc and then use
>>>>>>> ConcurrentHashMap
>>>>>>> and friends.
>>>>>>>
>>>>>>> But we gotta do it right....
>>>>>>>
>>>>>>> On 7 February 2011 19:57, Gregg Wonderly<gr...@wonderly.org>   wrote:
>>>>>>>
>>>>>>>   On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>   Sorry, I've got my nit-picking hat on...
>>>>>>>>
>>>>>>>>>
>>>>>>>>> The null check in the equals method isn't needed because you get a
>>>>>>>>> free one by using "instanceof".
>>>>>>>>>
>>>>>>>>> I know you've warned that the class is not thread-safe, but people
>>>>>>>>> often forget that doing a sysout of an object calls toString(). I'd
>>>>>>>>> still be tempted to whack some protection into toString to stop it
>>>>>>>>> blowing up when people start printing lots of debug messages.
>>>>>>>>>
>>>>>>>>> Other than that it looks good to me.
>>>>>>>>>
>>>>>>>>>   I'd suggest that it would be easy enough to just use
>>>>>>>>>
>>>>>>>> ConcurrentHashMap
>>>>>>>> instead of HashSet if we are going to move to Java 5 anyway. Then,
>>>>>>>> you
>>>>>>>> don't have to worry about concurrency issues with toString() etc.
>>>>>>>>
>>>>>>>> Gregg Wonderly
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>>>>>>>
>>>>>>>>   wrote:
>>>>>>>>>
>>>>>>>>>   Dan Creswell wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>   Solid - probably ought to be a constructor that doesn't add
>>>>>>>>>> defaults
>>>>>>>>>>
>>>>>>>>>>> too.....and maybe a note about thread safety on add.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>   The finished article.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> /*
>>>>>>>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>>>>>>>> * or more contributor license agreements. See the NOTICE file
>>>>>>>>>> * distributed with this work for additional information
>>>>>>>>>> * regarding copyright ownership. The ASF licenses this file
>>>>>>>>>> * to you under the Apache License, Version 2.0 (the
>>>>>>>>>> * "License"); you may not use this file except in compliance
>>>>>>>>>> * with the License. You may obtain a copy of the License at
>>>>>>>>>> *
>>>>>>>>>> * http://www.apache.org/licenses/LICENSE-2.0
>>>>>>>>>> *
>>>>>>>>>> * Unless required by applicable law or agreed to in writing,
>>>>>>>>>> software
>>>>>>>>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>>>>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>>>>>>> implied.
>>>>>>>>>> * See the License for the specific language governing permissions
>>>>>>>>>> and
>>>>>>>>>> * limitations under the License.
>>>>>>>>>> */
>>>>>>>>>>
>>>>>>>>>> package org.apache.river.api.lookup;
>>>>>>>>>>
>>>>>>>>>> import java.util.HashSet;
>>>>>>>>>> import java.util.Iterator;
>>>>>>>>>> import java.util.Set;
>>>>>>>>>> import net.jini.lookup.entry.Address;
>>>>>>>>>> import net.jini.lookup.entry.Comment;
>>>>>>>>>> import net.jini.lookup.entry.Location;
>>>>>>>>>> import net.jini.lookup.entry.Name;
>>>>>>>>>> import net.jini.lookup.entry.ServiceInfo;
>>>>>>>>>> import net.jini.lookup.entry.Status;
>>>>>>>>>> import net.jini.lookup.entry.UIDescriptor;
>>>>>>>>>>
>>>>>>>>>> /**
>>>>>>>>>> * A little builder utility class that creates an array of Entry
>>>>>>>>>> classes
>>>>>>>>>> to
>>>>>>>>>> * be used as a parameter for StreamServiceRegistrar. All the jini
>>>>>>>>>> platform
>>>>>>>>>> * Entry's are included by default.
>>>>>>>>>> *
>>>>>>>>>> * Note: This class is not threadsafe, use external synchronization
>>>>>>>>>> if
>>>>>>>>>> required.
>>>>>>>>>> *
>>>>>>>>>> * Suggested by Dan Creswell.
>>>>>>>>>> * @author peter
>>>>>>>>>> */
>>>>>>>>>> public class DefaultEntries {
>>>>>>>>>> private final Set<Class>   entrys;
>>>>>>>>>> public DefaultEntries() {
>>>>>>>>>> entrys = new HashSet<Class>(16);
>>>>>>>>>> }
>>>>>>>>>> /**
>>>>>>>>>> * Add an Entry class.
>>>>>>>>>> * @param cl - class
>>>>>>>>>> * @return this
>>>>>>>>>> */
>>>>>>>>>> public DefaultEntries add(Class cl){
>>>>>>>>>> entrys.add(cl);
>>>>>>>>>> return this;
>>>>>>>>>> }
>>>>>>>>>> /**
>>>>>>>>>> * All all the Jini Platform Entry's
>>>>>>>>>> * @return
>>>>>>>>>> */
>>>>>>>>>> public DefaultEntries addPlatformEntries(){
>>>>>>>>>> add(Comment.class);
>>>>>>>>>> add(Location.class);
>>>>>>>>>> add(Name.class);
>>>>>>>>>> add(ServiceInfo.class);
>>>>>>>>>> add(Status.class);
>>>>>>>>>> add(UIDescriptor.class);
>>>>>>>>>> add(Address.class);
>>>>>>>>>> return this;
>>>>>>>>>> }
>>>>>>>>>> /**
>>>>>>>>>> * Remove all Entry's
>>>>>>>>>> */
>>>>>>>>>> public void reset(){
>>>>>>>>>> entrys.clear();
>>>>>>>>>> }
>>>>>>>>>> /**
>>>>>>>>>> * Generate a new array containing all Entry's added since last
>>>>>>>>>> reset.
>>>>>>>>>> * @return
>>>>>>>>>> */
>>>>>>>>>> public Class[] getEntries(){
>>>>>>>>>> return entrys.toArray(new Class[entrys.size()]);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> @Override
>>>>>>>>>> public int hashCode() {
>>>>>>>>>> int hash = 3;
>>>>>>>>>> hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>>>>>>>>> 0);
>>>>>>>>>> return hash;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> @Override
>>>>>>>>>> public boolean equals(Object o){
>>>>>>>>>> if (o == null) return false;
>>>>>>>>>> if (o instanceof DefaultEntries){
>>>>>>>>>> if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>>>>>>>> }
>>>>>>>>>> return false;
>>>>>>>>>> }
>>>>>>>>>> @Override
>>>>>>>>>> public String toString(){
>>>>>>>>>> String newline = System.getProperty("line.separator");
>>>>>>>>>> StringBuilder sb = new StringBuilder(256);
>>>>>>>>>> sb.append("DefaultEntries:");
>>>>>>>>>> sb.append(newline);
>>>>>>>>>> Iterator<Class>   it = entrys.iterator();
>>>>>>>>>> while (it.hasNext()){
>>>>>>>>>> sb.append(it.next().getName());
>>>>>>>>>> sb.append(newline);
>>>>>>>>>> }
>>>>>>>>>> return sb.toString();
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

No friction, simplest thing that could possibly work is the only thing
running through my head.

I'm happy to see CHM used if people deem it appropriate - as is so often the
case, I'm not writing the code so my opinion should only carry so much
weight.

On 8 February 2011 15:12, Gregg Wonderly <gr...@wonderly.org> wrote:

> On 2/8/2011 9:00 AM, Dan Creswell wrote:
>
>> It's all about expected usage patterns.
>>
>> Locking in this case will be per instance and the likelihood of contention
>> is low given no one's going to call add a lot or indeed toString a lot in
>> a
>> short space of time.
>>
>> So given:
>>
>> Define how concurrency is handled then implement
>>
>> And assuming it's in line with the above contention model I'd say
>> synchronized would be adequate.
>>
>
> I believe this is the case.  But, I also know that it can come back to
> bite.
>
> CHM solves it without synchronized, and that, I believe is a better choice
> because it eliminates the problem.  Any additional methods or usage patterns
> that fall out of the life cycle of this object will not have any new or
> different behavior or synchronization requirements.
>
> Is there an attribute of CHM that you find unattractive Dan?  I'm not sure
> I understand the friction I am sensing.
>
> Gregg
>
>
>  On 8 February 2011 14:54, Gregg Wonderly<gr...@wonderly.org>  wrote:
>>
>>  On 2/7/2011 5:28 PM, Peter Firmstone wrote:
>>>
>>>  Do we really need a concurrent version of this class? How often will
>>>> Entry
>>>> classes change?
>>>>
>>>>
>>> My choice is always to consider concurrency first, unless there is some
>>> element of performance that this impacts.  The simple fact is that there
>>> is
>>> a toString() that traverses the contents.  So, you either must
>>> synchronize
>>> all access to the contents, or use a CHM which solves the problem without
>>> any future concurrency surprises.
>>>
>>> I'm really serious about this, because of how I've encountered tons of
>>> "synchronized" blocks in river which introduce huge latency in
>>> "processing"
>>> because they cause everyone to wait in line for stuff which very often is
>>> quite independent.
>>>
>>> Yes, this structure is mostly one time mutated.  But, its design exposes
>>> a
>>> continuous mutation capability.  Thus, any "read" of the contents has to
>>> take that into account.
>>>
>>> Gregg Wonderly
>>>
>>>
>>>  Gregg Wonderly wrote:
>>>
>>>>
>>>>  Yes definitely the case. CHM just makes it a lot easier to not worry
>>>>> about map
>>>>> and set data structures in terms of concurrent access of mutating data.
>>>>> Set
>>>>> and HashSet are useful, but not always in concurrent mutating data
>>>>> management.
>>>>>
>>>>> I still prefer Vector over ArrayList just because it eliminates
>>>>> iteration
>>>>> problems that alway require the same lines of code to resolve.
>>>>>
>>>>> Gregg
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On Feb 7, 2011, at 2:08 PM, Dan Creswell<da...@gmail.com>
>>>>>  wrote:
>>>>>
>>>>>  All true, still need to document the concurrent contract for this
>>>>> class
>>>>>
>>>>>> and
>>>>>> implement accordingly.
>>>>>>
>>>>>> i.e. We can say "toString" is thread safe etc and then use
>>>>>> ConcurrentHashMap
>>>>>> and friends.
>>>>>>
>>>>>> But we gotta do it right....
>>>>>>
>>>>>> On 7 February 2011 19:57, Gregg Wonderly<gr...@wonderly.org>  wrote:
>>>>>>
>>>>>>  On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>>>>>>
>>>>>>>
>>>>>>>  Sorry, I've got my nit-picking hat on...
>>>>>>>
>>>>>>>>
>>>>>>>> The null check in the equals method isn't needed because you get a
>>>>>>>> free one by using "instanceof".
>>>>>>>>
>>>>>>>> I know you've warned that the class is not thread-safe, but people
>>>>>>>> often forget that doing a sysout of an object calls toString(). I'd
>>>>>>>> still be tempted to whack some protection into toString to stop it
>>>>>>>> blowing up when people start printing lots of debug messages.
>>>>>>>>
>>>>>>>> Other than that it looks good to me.
>>>>>>>>
>>>>>>>>  I'd suggest that it would be easy enough to just use
>>>>>>>>
>>>>>>> ConcurrentHashMap
>>>>>>> instead of HashSet if we are going to move to Java 5 anyway. Then,
>>>>>>> you
>>>>>>> don't have to worry about concurrency issues with toString() etc.
>>>>>>>
>>>>>>> Gregg Wonderly
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>>>>>>
>>>>>>>  wrote:
>>>>>>>>
>>>>>>>>  Dan Creswell wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>  Solid - probably ought to be a constructor that doesn't add
>>>>>>>>> defaults
>>>>>>>>>
>>>>>>>>>> too.....and maybe a note about thread safety on add.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  The finished article.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> /*
>>>>>>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>>>>>>> * or more contributor license agreements. See the NOTICE file
>>>>>>>>> * distributed with this work for additional information
>>>>>>>>> * regarding copyright ownership. The ASF licenses this file
>>>>>>>>> * to you under the Apache License, Version 2.0 (the
>>>>>>>>> * "License"); you may not use this file except in compliance
>>>>>>>>> * with the License. You may obtain a copy of the License at
>>>>>>>>> *
>>>>>>>>> * http://www.apache.org/licenses/LICENSE-2.0
>>>>>>>>> *
>>>>>>>>> * Unless required by applicable law or agreed to in writing,
>>>>>>>>> software
>>>>>>>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>>>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>>>>>> implied.
>>>>>>>>> * See the License for the specific language governing permissions
>>>>>>>>> and
>>>>>>>>> * limitations under the License.
>>>>>>>>> */
>>>>>>>>>
>>>>>>>>> package org.apache.river.api.lookup;
>>>>>>>>>
>>>>>>>>> import java.util.HashSet;
>>>>>>>>> import java.util.Iterator;
>>>>>>>>> import java.util.Set;
>>>>>>>>> import net.jini.lookup.entry.Address;
>>>>>>>>> import net.jini.lookup.entry.Comment;
>>>>>>>>> import net.jini.lookup.entry.Location;
>>>>>>>>> import net.jini.lookup.entry.Name;
>>>>>>>>> import net.jini.lookup.entry.ServiceInfo;
>>>>>>>>> import net.jini.lookup.entry.Status;
>>>>>>>>> import net.jini.lookup.entry.UIDescriptor;
>>>>>>>>>
>>>>>>>>> /**
>>>>>>>>> * A little builder utility class that creates an array of Entry
>>>>>>>>> classes
>>>>>>>>> to
>>>>>>>>> * be used as a parameter for StreamServiceRegistrar. All the jini
>>>>>>>>> platform
>>>>>>>>> * Entry's are included by default.
>>>>>>>>> *
>>>>>>>>> * Note: This class is not threadsafe, use external synchronization
>>>>>>>>> if
>>>>>>>>> required.
>>>>>>>>> *
>>>>>>>>> * Suggested by Dan Creswell.
>>>>>>>>> * @author peter
>>>>>>>>> */
>>>>>>>>> public class DefaultEntries {
>>>>>>>>> private final Set<Class>  entrys;
>>>>>>>>> public DefaultEntries() {
>>>>>>>>> entrys = new HashSet<Class>(16);
>>>>>>>>> }
>>>>>>>>> /**
>>>>>>>>> * Add an Entry class.
>>>>>>>>> * @param cl - class
>>>>>>>>> * @return this
>>>>>>>>> */
>>>>>>>>> public DefaultEntries add(Class cl){
>>>>>>>>> entrys.add(cl);
>>>>>>>>> return this;
>>>>>>>>> }
>>>>>>>>> /**
>>>>>>>>> * All all the Jini Platform Entry's
>>>>>>>>> * @return
>>>>>>>>> */
>>>>>>>>> public DefaultEntries addPlatformEntries(){
>>>>>>>>> add(Comment.class);
>>>>>>>>> add(Location.class);
>>>>>>>>> add(Name.class);
>>>>>>>>> add(ServiceInfo.class);
>>>>>>>>> add(Status.class);
>>>>>>>>> add(UIDescriptor.class);
>>>>>>>>> add(Address.class);
>>>>>>>>> return this;
>>>>>>>>> }
>>>>>>>>> /**
>>>>>>>>> * Remove all Entry's
>>>>>>>>> */
>>>>>>>>> public void reset(){
>>>>>>>>> entrys.clear();
>>>>>>>>> }
>>>>>>>>> /**
>>>>>>>>> * Generate a new array containing all Entry's added since last
>>>>>>>>> reset.
>>>>>>>>> * @return
>>>>>>>>> */
>>>>>>>>> public Class[] getEntries(){
>>>>>>>>> return entrys.toArray(new Class[entrys.size()]);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> @Override
>>>>>>>>> public int hashCode() {
>>>>>>>>> int hash = 3;
>>>>>>>>> hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>>>>>>>> 0);
>>>>>>>>> return hash;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> @Override
>>>>>>>>> public boolean equals(Object o){
>>>>>>>>> if (o == null) return false;
>>>>>>>>> if (o instanceof DefaultEntries){
>>>>>>>>> if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>>>>>>> }
>>>>>>>>> return false;
>>>>>>>>> }
>>>>>>>>> @Override
>>>>>>>>> public String toString(){
>>>>>>>>> String newline = System.getProperty("line.separator");
>>>>>>>>> StringBuilder sb = new StringBuilder(256);
>>>>>>>>> sb.append("DefaultEntries:");
>>>>>>>>> sb.append(newline);
>>>>>>>>> Iterator<Class>  it = entrys.iterator();
>>>>>>>>> while (it.hasNext()){
>>>>>>>>> sb.append(it.next().getName());
>>>>>>>>> sb.append(newline);
>>>>>>>>> }
>>>>>>>>> return sb.toString();
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

On 2/8/2011 9:00 AM, Dan Creswell wrote:
> It's all about expected usage patterns.
>
> Locking in this case will be per instance and the likelihood of contention
> is low given no one's going to call add a lot or indeed toString a lot in a
> short space of time.
>
> So given:
>
> Define how concurrency is handled then implement
>
> And assuming it's in line with the above contention model I'd say
> synchronized would be adequate.

I believe this is the case.  But, I also know that it can come back to bite.

CHM solves it without synchronized, and that, I believe is a better choice 
because it eliminates the problem.  Any additional methods or usage patterns 
that fall out of the life cycle of this object will not have any new or 
different behavior or synchronization requirements.

Is there an attribute of CHM that you find unattractive Dan?  I'm not sure I 
understand the friction I am sensing.

Gregg

> On 8 February 2011 14:54, Gregg Wonderly<gr...@wonderly.org>  wrote:
>
>> On 2/7/2011 5:28 PM, Peter Firmstone wrote:
>>
>>> Do we really need a concurrent version of this class? How often will Entry
>>> classes change?
>>>
>>
>> My choice is always to consider concurrency first, unless there is some
>> element of performance that this impacts.  The simple fact is that there is
>> a toString() that traverses the contents.  So, you either must synchronize
>> all access to the contents, or use a CHM which solves the problem without
>> any future concurrency surprises.
>>
>> I'm really serious about this, because of how I've encountered tons of
>> "synchronized" blocks in river which introduce huge latency in "processing"
>> because they cause everyone to wait in line for stuff which very often is
>> quite independent.
>>
>> Yes, this structure is mostly one time mutated.  But, its design exposes a
>> continuous mutation capability.  Thus, any "read" of the contents has to
>> take that into account.
>>
>> Gregg Wonderly
>>
>>
>>   Gregg Wonderly wrote:
>>>
>>>> Yes definitely the case. CHM just makes it a lot easier to not worry
>>>> about map
>>>> and set data structures in terms of concurrent access of mutating data.
>>>> Set
>>>> and HashSet are useful, but not always in concurrent mutating data
>>>> management.
>>>>
>>>> I still prefer Vector over ArrayList just because it eliminates iteration
>>>> problems that alway require the same lines of code to resolve.
>>>>
>>>> Gregg
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Feb 7, 2011, at 2:08 PM, Dan Creswell<da...@gmail.com>  wrote:
>>>>
>>>>   All true, still need to document the concurrent contract for this class
>>>>> and
>>>>> implement accordingly.
>>>>>
>>>>> i.e. We can say "toString" is thread safe etc and then use
>>>>> ConcurrentHashMap
>>>>> and friends.
>>>>>
>>>>> But we gotta do it right....
>>>>>
>>>>> On 7 February 2011 19:57, Gregg Wonderly<gr...@wonderly.org>  wrote:
>>>>>
>>>>>   On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>>>>>>
>>>>>>   Sorry, I've got my nit-picking hat on...
>>>>>>>
>>>>>>> The null check in the equals method isn't needed because you get a
>>>>>>> free one by using "instanceof".
>>>>>>>
>>>>>>> I know you've warned that the class is not thread-safe, but people
>>>>>>> often forget that doing a sysout of an object calls toString(). I'd
>>>>>>> still be tempted to whack some protection into toString to stop it
>>>>>>> blowing up when people start printing lots of debug messages.
>>>>>>>
>>>>>>> Other than that it looks good to me.
>>>>>>>
>>>>>>>   I'd suggest that it would be easy enough to just use
>>>>>> ConcurrentHashMap
>>>>>> instead of HashSet if we are going to move to Java 5 anyway. Then, you
>>>>>> don't have to worry about concurrency issues with toString() etc.
>>>>>>
>>>>>> Gregg Wonderly
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>   Dan Creswell wrote:
>>>>>>>>
>>>>>>>>   Solid - probably ought to be a constructor that doesn't add defaults
>>>>>>>>> too.....and maybe a note about thread safety on add.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>   The finished article.
>>>>>>>>
>>>>>>>> /*
>>>>>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>>>>>> * or more contributor license agreements. See the NOTICE file
>>>>>>>> * distributed with this work for additional information
>>>>>>>> * regarding copyright ownership. The ASF licenses this file
>>>>>>>> * to you under the Apache License, Version 2.0 (the
>>>>>>>> * "License"); you may not use this file except in compliance
>>>>>>>> * with the License. You may obtain a copy of the License at
>>>>>>>> *
>>>>>>>> * http://www.apache.org/licenses/LICENSE-2.0
>>>>>>>> *
>>>>>>>> * Unless required by applicable law or agreed to in writing, software
>>>>>>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>>>>> implied.
>>>>>>>> * See the License for the specific language governing permissions and
>>>>>>>> * limitations under the License.
>>>>>>>> */
>>>>>>>>
>>>>>>>> package org.apache.river.api.lookup;
>>>>>>>>
>>>>>>>> import java.util.HashSet;
>>>>>>>> import java.util.Iterator;
>>>>>>>> import java.util.Set;
>>>>>>>> import net.jini.lookup.entry.Address;
>>>>>>>> import net.jini.lookup.entry.Comment;
>>>>>>>> import net.jini.lookup.entry.Location;
>>>>>>>> import net.jini.lookup.entry.Name;
>>>>>>>> import net.jini.lookup.entry.ServiceInfo;
>>>>>>>> import net.jini.lookup.entry.Status;
>>>>>>>> import net.jini.lookup.entry.UIDescriptor;
>>>>>>>>
>>>>>>>> /**
>>>>>>>> * A little builder utility class that creates an array of Entry
>>>>>>>> classes
>>>>>>>> to
>>>>>>>> * be used as a parameter for StreamServiceRegistrar. All the jini
>>>>>>>> platform
>>>>>>>> * Entry's are included by default.
>>>>>>>> *
>>>>>>>> * Note: This class is not threadsafe, use external synchronization if
>>>>>>>> required.
>>>>>>>> *
>>>>>>>> * Suggested by Dan Creswell.
>>>>>>>> * @author peter
>>>>>>>> */
>>>>>>>> public class DefaultEntries {
>>>>>>>> private final Set<Class>  entrys;
>>>>>>>> public DefaultEntries() {
>>>>>>>> entrys = new HashSet<Class>(16);
>>>>>>>> }
>>>>>>>> /**
>>>>>>>> * Add an Entry class.
>>>>>>>> * @param cl - class
>>>>>>>> * @return this
>>>>>>>> */
>>>>>>>> public DefaultEntries add(Class cl){
>>>>>>>> entrys.add(cl);
>>>>>>>> return this;
>>>>>>>> }
>>>>>>>> /**
>>>>>>>> * All all the Jini Platform Entry's
>>>>>>>> * @return
>>>>>>>> */
>>>>>>>> public DefaultEntries addPlatformEntries(){
>>>>>>>> add(Comment.class);
>>>>>>>> add(Location.class);
>>>>>>>> add(Name.class);
>>>>>>>> add(ServiceInfo.class);
>>>>>>>> add(Status.class);
>>>>>>>> add(UIDescriptor.class);
>>>>>>>> add(Address.class);
>>>>>>>> return this;
>>>>>>>> }
>>>>>>>> /**
>>>>>>>> * Remove all Entry's
>>>>>>>> */
>>>>>>>> public void reset(){
>>>>>>>> entrys.clear();
>>>>>>>> }
>>>>>>>> /**
>>>>>>>> * Generate a new array containing all Entry's added since last reset.
>>>>>>>> * @return
>>>>>>>> */
>>>>>>>> public Class[] getEntries(){
>>>>>>>> return entrys.toArray(new Class[entrys.size()]);
>>>>>>>> }
>>>>>>>>
>>>>>>>> @Override
>>>>>>>> public int hashCode() {
>>>>>>>> int hash = 3;
>>>>>>>> hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>>>>>>> 0);
>>>>>>>> return hash;
>>>>>>>> }
>>>>>>>>
>>>>>>>> @Override
>>>>>>>> public boolean equals(Object o){
>>>>>>>> if (o == null) return false;
>>>>>>>> if (o instanceof DefaultEntries){
>>>>>>>> if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>>>>>> }
>>>>>>>> return false;
>>>>>>>> }
>>>>>>>> @Override
>>>>>>>> public String toString(){
>>>>>>>> String newline = System.getProperty("line.separator");
>>>>>>>> StringBuilder sb = new StringBuilder(256);
>>>>>>>> sb.append("DefaultEntries:");
>>>>>>>> sb.append(newline);
>>>>>>>> Iterator<Class>  it = entrys.iterator();
>>>>>>>> while (it.hasNext()){
>>>>>>>> sb.append(it.next().getName());
>>>>>>>> sb.append(newline);
>>>>>>>> }
>>>>>>>> return sb.toString();
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>
>>>
>>>
>>
>

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

It's all about expected usage patterns.

Locking in this case will be per instance and the likelihood of contention
is low given no one's going to call add a lot or indeed toString a lot in a
short space of time.

So given:

Define how concurrency is handled then implement

And assuming it's in line with the above contention model I'd say
synchronized would be adequate.

On 8 February 2011 14:54, Gregg Wonderly <gr...@wonderly.org> wrote:

> On 2/7/2011 5:28 PM, Peter Firmstone wrote:
>
>> Do we really need a concurrent version of this class? How often will Entry
>> classes change?
>>
>
> My choice is always to consider concurrency first, unless there is some
> element of performance that this impacts.  The simple fact is that there is
> a toString() that traverses the contents.  So, you either must synchronize
> all access to the contents, or use a CHM which solves the problem without
> any future concurrency surprises.
>
> I'm really serious about this, because of how I've encountered tons of
> "synchronized" blocks in river which introduce huge latency in "processing"
> because they cause everyone to wait in line for stuff which very often is
> quite independent.
>
> Yes, this structure is mostly one time mutated.  But, its design exposes a
> continuous mutation capability.  Thus, any "read" of the contents has to
> take that into account.
>
> Gregg Wonderly
>
>
>  Gregg Wonderly wrote:
>>
>>> Yes definitely the case. CHM just makes it a lot easier to not worry
>>> about map
>>> and set data structures in terms of concurrent access of mutating data.
>>> Set
>>> and HashSet are useful, but not always in concurrent mutating data
>>> management.
>>>
>>> I still prefer Vector over ArrayList just because it eliminates iteration
>>> problems that alway require the same lines of code to resolve.
>>>
>>> Gregg
>>>
>>> Sent from my iPhone
>>>
>>> On Feb 7, 2011, at 2:08 PM, Dan Creswell <da...@gmail.com> wrote:
>>>
>>>  All true, still need to document the concurrent contract for this class
>>>> and
>>>> implement accordingly.
>>>>
>>>> i.e. We can say "toString" is thread safe etc and then use
>>>> ConcurrentHashMap
>>>> and friends.
>>>>
>>>> But we gotta do it right....
>>>>
>>>> On 7 February 2011 19:57, Gregg Wonderly <gr...@wonderly.org> wrote:
>>>>
>>>>  On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>>>>>
>>>>>  Sorry, I've got my nit-picking hat on...
>>>>>>
>>>>>> The null check in the equals method isn't needed because you get a
>>>>>> free one by using "instanceof".
>>>>>>
>>>>>> I know you've warned that the class is not thread-safe, but people
>>>>>> often forget that doing a sysout of an object calls toString(). I'd
>>>>>> still be tempted to whack some protection into toString to stop it
>>>>>> blowing up when people start printing lots of debug messages.
>>>>>>
>>>>>> Other than that it looks good to me.
>>>>>>
>>>>>>  I'd suggest that it would be easy enough to just use
>>>>> ConcurrentHashMap
>>>>> instead of HashSet if we are going to move to Java 5 anyway. Then, you
>>>>> don't have to worry about concurrency issues with toString() etc.
>>>>>
>>>>> Gregg Wonderly
>>>>>
>>>>>
>>>>> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>  Dan Creswell wrote:
>>>>>>>
>>>>>>>  Solid - probably ought to be a constructor that doesn't add defaults
>>>>>>>> too.....and maybe a note about thread safety on add.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>  The finished article.
>>>>>>>
>>>>>>> /*
>>>>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>>>>> * or more contributor license agreements. See the NOTICE file
>>>>>>> * distributed with this work for additional information
>>>>>>> * regarding copyright ownership. The ASF licenses this file
>>>>>>> * to you under the Apache License, Version 2.0 (the
>>>>>>> * "License"); you may not use this file except in compliance
>>>>>>> * with the License. You may obtain a copy of the License at
>>>>>>> *
>>>>>>> * http://www.apache.org/licenses/LICENSE-2.0
>>>>>>> *
>>>>>>> * Unless required by applicable law or agreed to in writing, software
>>>>>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>>>> implied.
>>>>>>> * See the License for the specific language governing permissions and
>>>>>>> * limitations under the License.
>>>>>>> */
>>>>>>>
>>>>>>> package org.apache.river.api.lookup;
>>>>>>>
>>>>>>> import java.util.HashSet;
>>>>>>> import java.util.Iterator;
>>>>>>> import java.util.Set;
>>>>>>> import net.jini.lookup.entry.Address;
>>>>>>> import net.jini.lookup.entry.Comment;
>>>>>>> import net.jini.lookup.entry.Location;
>>>>>>> import net.jini.lookup.entry.Name;
>>>>>>> import net.jini.lookup.entry.ServiceInfo;
>>>>>>> import net.jini.lookup.entry.Status;
>>>>>>> import net.jini.lookup.entry.UIDescriptor;
>>>>>>>
>>>>>>> /**
>>>>>>> * A little builder utility class that creates an array of Entry
>>>>>>> classes
>>>>>>> to
>>>>>>> * be used as a parameter for StreamServiceRegistrar. All the jini
>>>>>>> platform
>>>>>>> * Entry's are included by default.
>>>>>>> *
>>>>>>> * Note: This class is not threadsafe, use external synchronization if
>>>>>>> required.
>>>>>>> *
>>>>>>> * Suggested by Dan Creswell.
>>>>>>> * @author peter
>>>>>>> */
>>>>>>> public class DefaultEntries {
>>>>>>> private final Set<Class> entrys;
>>>>>>> public DefaultEntries() {
>>>>>>> entrys = new HashSet<Class>(16);
>>>>>>> }
>>>>>>> /**
>>>>>>> * Add an Entry class.
>>>>>>> * @param cl - class
>>>>>>> * @return this
>>>>>>> */
>>>>>>> public DefaultEntries add(Class cl){
>>>>>>> entrys.add(cl);
>>>>>>> return this;
>>>>>>> }
>>>>>>> /**
>>>>>>> * All all the Jini Platform Entry's
>>>>>>> * @return
>>>>>>> */
>>>>>>> public DefaultEntries addPlatformEntries(){
>>>>>>> add(Comment.class);
>>>>>>> add(Location.class);
>>>>>>> add(Name.class);
>>>>>>> add(ServiceInfo.class);
>>>>>>> add(Status.class);
>>>>>>> add(UIDescriptor.class);
>>>>>>> add(Address.class);
>>>>>>> return this;
>>>>>>> }
>>>>>>> /**
>>>>>>> * Remove all Entry's
>>>>>>> */
>>>>>>> public void reset(){
>>>>>>> entrys.clear();
>>>>>>> }
>>>>>>> /**
>>>>>>> * Generate a new array containing all Entry's added since last reset.
>>>>>>> * @return
>>>>>>> */
>>>>>>> public Class[] getEntries(){
>>>>>>> return entrys.toArray(new Class[entrys.size()]);
>>>>>>> }
>>>>>>>
>>>>>>> @Override
>>>>>>> public int hashCode() {
>>>>>>> int hash = 3;
>>>>>>> hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>>>>>> 0);
>>>>>>> return hash;
>>>>>>> }
>>>>>>>
>>>>>>> @Override
>>>>>>> public boolean equals(Object o){
>>>>>>> if (o == null) return false;
>>>>>>> if (o instanceof DefaultEntries){
>>>>>>> if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>>>>> }
>>>>>>> return false;
>>>>>>> }
>>>>>>> @Override
>>>>>>> public String toString(){
>>>>>>> String newline = System.getProperty("line.separator");
>>>>>>> StringBuilder sb = new StringBuilder(256);
>>>>>>> sb.append("DefaultEntries:");
>>>>>>> sb.append(newline);
>>>>>>> Iterator<Class> it = entrys.iterator();
>>>>>>> while (it.hasNext()){
>>>>>>> sb.append(it.next().getName());
>>>>>>> sb.append(newline);
>>>>>>> }
>>>>>>> return sb.toString();
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>>
>>>
>>
>>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

There we go, bells & whistles included.

Cheers,

Peter.


/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.river.api.lookup;

import java.util.HashSet;
import java.util.Iterator;
import java.util.Set;
import net.jini.lookup.entry.Address;
import net.jini.lookup.entry.Comment;
import net.jini.lookup.entry.Location;
import net.jini.lookup.entry.Name;
import net.jini.lookup.entry.ServiceInfo;
import net.jini.lookup.entry.Status;
import net.jini.lookup.entry.UIDescriptor;
import org.apache.river.impl.util.ConcurrentCollections;

/**
 * A little builder utility class that creates an array of Entry classes to
 * be used as a parameter for StreamServiceRegistrar.
 *
 * Note: This class is thread safe.
 *
 * The toString method will return a snapshot of the Entry classes 
contained at
 * the time of the toString method call, the String returned will be a
 * space delimited list of all Entry classes.
 *
 *
 * Suggested by Dan Creswell.
 * @author peter
 */
public class DefaultEntries {
    private final Set<Class> entrys;
   
    public DefaultEntries() {
        Set<Class> set = new HashSet<Class>(16);
        entrys = ConcurrentCollections.multiReadSet(set);
    }
    /**
     * Add an Entry class.
     * @param cl - class
     * @return this
     */
    public DefaultEntries add(Class cl){
        entrys.add(cl);
        return this;
    }
    /**
     * All all the Jini Platform Entry's
     * @return
     */
    public DefaultEntries addPlatformEntries(){
        add(Comment.class);
        add(Location.class);
        add(Name.class);
        add(ServiceInfo.class);
        add(Status.class);
        add(UIDescriptor.class);
        add(Address.class);
        return this;
    }
    /**
     * Remove all Entry's
     */
    public void reset(){
        entrys.clear();
    }
   
    /**
     * Generate a new array containing all Entry's added since last reset.
     *
     * @return
     */
    public Class[] getEntries(){
        /* An empty array is passed in, it will be replaced by the correct
         * size array. Checking the Set size first, then creating an 
array of
         * sufficient size would not be atomic and may lead to an array
         * filled with null's if reset was called between size and
         * passing the array into the entrys Set.
         */
        return entrys.toArray(new Class[0]);
    }

    @Override
    public int hashCode() {
        int hash = 3;
        hash = 29 * hash + this.entrys.hashCode();
        return hash;
    }

    @Override
    public boolean equals(Object o){
        if (o instanceof DefaultEntries){
            if (entrys.equals(((DefaultEntries)o).entrys)) return true;
        }
        return false;
    }
   
    @Override
    public String toString(){
        StringBuilder sb = new StringBuilder(256);
        sb.append("DefaultEntries:");
        Iterator<Class> it = entrys.iterator();
        while (it.hasNext()){
            sb.append(" ");
            sb.append(it.next().getSimpleName());
        }
        return sb.toString();
    }
}

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

On 2/7/2011 5:28 PM, Peter Firmstone wrote:
> Do we really need a concurrent version of this class? How often will Entry
> classes change?

My choice is always to consider concurrency first, unless there is some element 
of performance that this impacts.  The simple fact is that there is a toString() 
that traverses the contents.  So, you either must synchronize all access to the 
contents, or use a CHM which solves the problem without any future concurrency 
surprises.

I'm really serious about this, because of how I've encountered tons of 
"synchronized" blocks in river which introduce huge latency in "processing" 
because they cause everyone to wait in line for stuff which very often is quite 
independent.

Yes, this structure is mostly one time mutated.  But, its design exposes a 
continuous mutation capability.  Thus, any "read" of the contents has to take 
that into account.

Gregg Wonderly

> Gregg Wonderly wrote:
>> Yes definitely the case. CHM just makes it a lot easier to not worry about map
>> and set data structures in terms of concurrent access of mutating data. Set
>> and HashSet are useful, but not always in concurrent mutating data management.
>>
>> I still prefer Vector over ArrayList just because it eliminates iteration
>> problems that alway require the same lines of code to resolve.
>>
>> Gregg
>>
>> Sent from my iPhone
>>
>> On Feb 7, 2011, at 2:08 PM, Dan Creswell <da...@gmail.com> wrote:
>>
>>> All true, still need to document the concurrent contract for this class and
>>> implement accordingly.
>>>
>>> i.e. We can say "toString" is thread safe etc and then use ConcurrentHashMap
>>> and friends.
>>>
>>> But we gotta do it right....
>>>
>>> On 7 February 2011 19:57, Gregg Wonderly <gr...@wonderly.org> wrote:
>>>
>>>> On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>>>>
>>>>> Sorry, I've got my nit-picking hat on...
>>>>>
>>>>> The null check in the equals method isn't needed because you get a
>>>>> free one by using "instanceof".
>>>>>
>>>>> I know you've warned that the class is not thread-safe, but people
>>>>> often forget that doing a sysout of an object calls toString(). I'd
>>>>> still be tempted to whack some protection into toString to stop it
>>>>> blowing up when people start printing lots of debug messages.
>>>>>
>>>>> Other than that it looks good to me.
>>>>>
>>>> I'd suggest that it would be easy enough to just use ConcurrentHashMap
>>>> instead of HashSet if we are going to move to Java 5 anyway. Then, you
>>>> don't have to worry about concurrency issues with toString() etc.
>>>>
>>>> Gregg Wonderly
>>>>
>>>>
>>>> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>>>> wrote:
>>>>>
>>>>>> Dan Creswell wrote:
>>>>>>
>>>>>>> Solid - probably ought to be a constructor that doesn't add defaults
>>>>>>> too.....and maybe a note about thread safety on add.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> The finished article.
>>>>>>
>>>>>> /*
>>>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>>>> * or more contributor license agreements. See the NOTICE file
>>>>>> * distributed with this work for additional information
>>>>>> * regarding copyright ownership. The ASF licenses this file
>>>>>> * to you under the Apache License, Version 2.0 (the
>>>>>> * "License"); you may not use this file except in compliance
>>>>>> * with the License. You may obtain a copy of the License at
>>>>>> *
>>>>>> * http://www.apache.org/licenses/LICENSE-2.0
>>>>>> *
>>>>>> * Unless required by applicable law or agreed to in writing, software
>>>>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>>> implied.
>>>>>> * See the License for the specific language governing permissions and
>>>>>> * limitations under the License.
>>>>>> */
>>>>>>
>>>>>> package org.apache.river.api.lookup;
>>>>>>
>>>>>> import java.util.HashSet;
>>>>>> import java.util.Iterator;
>>>>>> import java.util.Set;
>>>>>> import net.jini.lookup.entry.Address;
>>>>>> import net.jini.lookup.entry.Comment;
>>>>>> import net.jini.lookup.entry.Location;
>>>>>> import net.jini.lookup.entry.Name;
>>>>>> import net.jini.lookup.entry.ServiceInfo;
>>>>>> import net.jini.lookup.entry.Status;
>>>>>> import net.jini.lookup.entry.UIDescriptor;
>>>>>>
>>>>>> /**
>>>>>> * A little builder utility class that creates an array of Entry classes
>>>>>> to
>>>>>> * be used as a parameter for StreamServiceRegistrar. All the jini
>>>>>> platform
>>>>>> * Entry's are included by default.
>>>>>> *
>>>>>> * Note: This class is not threadsafe, use external synchronization if
>>>>>> required.
>>>>>> *
>>>>>> * Suggested by Dan Creswell.
>>>>>> * @author peter
>>>>>> */
>>>>>> public class DefaultEntries {
>>>>>> private final Set<Class> entrys;
>>>>>> public DefaultEntries() {
>>>>>> entrys = new HashSet<Class>(16);
>>>>>> }
>>>>>> /**
>>>>>> * Add an Entry class.
>>>>>> * @param cl - class
>>>>>> * @return this
>>>>>> */
>>>>>> public DefaultEntries add(Class cl){
>>>>>> entrys.add(cl);
>>>>>> return this;
>>>>>> }
>>>>>> /**
>>>>>> * All all the Jini Platform Entry's
>>>>>> * @return
>>>>>> */
>>>>>> public DefaultEntries addPlatformEntries(){
>>>>>> add(Comment.class);
>>>>>> add(Location.class);
>>>>>> add(Name.class);
>>>>>> add(ServiceInfo.class);
>>>>>> add(Status.class);
>>>>>> add(UIDescriptor.class);
>>>>>> add(Address.class);
>>>>>> return this;
>>>>>> }
>>>>>> /**
>>>>>> * Remove all Entry's
>>>>>> */
>>>>>> public void reset(){
>>>>>> entrys.clear();
>>>>>> }
>>>>>> /**
>>>>>> * Generate a new array containing all Entry's added since last reset.
>>>>>> * @return
>>>>>> */
>>>>>> public Class[] getEntries(){
>>>>>> return entrys.toArray(new Class[entrys.size()]);
>>>>>> }
>>>>>>
>>>>>> @Override
>>>>>> public int hashCode() {
>>>>>> int hash = 3;
>>>>>> hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>>>>> 0);
>>>>>> return hash;
>>>>>> }
>>>>>>
>>>>>> @Override
>>>>>> public boolean equals(Object o){
>>>>>> if (o == null) return false;
>>>>>> if (o instanceof DefaultEntries){
>>>>>> if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>>>> }
>>>>>> return false;
>>>>>> }
>>>>>> @Override
>>>>>> public String toString(){
>>>>>> String newline = System.getProperty("line.separator");
>>>>>> StringBuilder sb = new StringBuilder(256);
>>>>>> sb.append("DefaultEntries:");
>>>>>> sb.append(newline);
>>>>>> Iterator<Class> it = entrys.iterator();
>>>>>> while (it.hasNext()){
>>>>>> sb.append(it.next().getName());
>>>>>> sb.append(newline);
>>>>>> }
>>>>>> return sb.toString();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>
>>
>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Do we really need a concurrent version of this class?  How often will 
Entry classes change?

Gregg Wonderly wrote:
> Yes definitely the case.  CHM just makes it a lot easier to not worry about map and set data structures in terms of concurrent access of mutating data. Set and HashSet are useful, but not always in concurrent mutating data management.
>
> I still prefer Vector over ArrayList just because it eliminates iteration problems that alway require the same lines of code to resolve.
>
> Gregg
>
> Sent from my iPhone
>
> On Feb 7, 2011, at 2:08 PM, Dan Creswell <da...@gmail.com> wrote:
>
>   
>> All true, still need to document the concurrent contract for this class and
>> implement accordingly.
>>
>> i.e. We can say "toString" is thread safe etc and then use ConcurrentHashMap
>> and friends.
>>
>> But we gotta do it right....
>>
>> On 7 February 2011 19:57, Gregg Wonderly <gr...@wonderly.org> wrote:
>>
>>     
>>> On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>>>
>>>       
>>>> Sorry, I've got my nit-picking hat on...
>>>>
>>>> The null check in the equals method isn't needed because you get a
>>>> free one by using "instanceof".
>>>>
>>>> I know you've warned that the class is not thread-safe, but people
>>>> often forget that doing a sysout of an object calls toString().  I'd
>>>> still be tempted to whack some protection into toString to stop it
>>>> blowing up when people start printing lots of debug messages.
>>>>
>>>> Other than that it looks good to me.
>>>>
>>>>         
>>> I'd suggest that it would be easy enough to just use ConcurrentHashMap
>>> instead of HashSet if we are going to move to Java 5 anyway.  Then, you
>>> don't have to worry about concurrency issues with toString() etc.
>>>
>>> Gregg Wonderly
>>>
>>>
>>> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>>       
>>>> wrote:
>>>>
>>>>         
>>>>> Dan Creswell wrote:
>>>>>
>>>>>           
>>>>>> Solid - probably ought to be a constructor that doesn't add defaults
>>>>>> too.....and maybe a note about thread safety on add.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> The finished article.
>>>>>
>>>>> /*
>>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>>> * or more contributor license agreements.  See the NOTICE file
>>>>> * distributed with this work for additional information
>>>>> * regarding copyright ownership. The ASF licenses this file
>>>>> * to you under the Apache License, Version 2.0 (the
>>>>> * "License"); you may not use this file except in compliance
>>>>> * with the License. You may obtain a copy of the License at
>>>>> *
>>>>> *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> *
>>>>> * Unless required by applicable law or agreed to in writing, software
>>>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>> implied.
>>>>> * See the License for the specific language governing permissions and
>>>>> * limitations under the License.
>>>>> */
>>>>>
>>>>> package org.apache.river.api.lookup;
>>>>>
>>>>> import java.util.HashSet;
>>>>> import java.util.Iterator;
>>>>> import java.util.Set;
>>>>> import net.jini.lookup.entry.Address;
>>>>> import net.jini.lookup.entry.Comment;
>>>>> import net.jini.lookup.entry.Location;
>>>>> import net.jini.lookup.entry.Name;
>>>>> import net.jini.lookup.entry.ServiceInfo;
>>>>> import net.jini.lookup.entry.Status;
>>>>> import net.jini.lookup.entry.UIDescriptor;
>>>>>
>>>>> /**
>>>>> * A little builder utility class that creates an array of Entry classes
>>>>> to
>>>>> * be used as a parameter for StreamServiceRegistrar.  All the jini
>>>>> platform
>>>>> * Entry's are included by default.
>>>>> *
>>>>> * Note: This class is not threadsafe, use external synchronization if
>>>>> required.
>>>>> *
>>>>> * Suggested by Dan Creswell.
>>>>> * @author peter
>>>>> */
>>>>> public class DefaultEntries {
>>>>>  private final Set<Class>  entrys;
>>>>>    public DefaultEntries() {
>>>>>      entrys = new HashSet<Class>(16);
>>>>>  }
>>>>>  /**
>>>>>   * Add an Entry class.
>>>>>   * @param cl - class
>>>>>   * @return this
>>>>>   */
>>>>>  public DefaultEntries add(Class cl){
>>>>>      entrys.add(cl);
>>>>>      return this;
>>>>>  }
>>>>>  /**
>>>>>   * All all the Jini Platform Entry's
>>>>>   * @return
>>>>>   */
>>>>>  public DefaultEntries addPlatformEntries(){
>>>>>      add(Comment.class);
>>>>>      add(Location.class);
>>>>>      add(Name.class);
>>>>>      add(ServiceInfo.class);
>>>>>      add(Status.class);
>>>>>      add(UIDescriptor.class);
>>>>>      add(Address.class);
>>>>>      return this;
>>>>>  }
>>>>>  /**
>>>>>   * Remove all Entry's
>>>>>   */
>>>>>  public void reset(){
>>>>>      entrys.clear();
>>>>>  }
>>>>>    /**
>>>>>   * Generate a new array containing all Entry's added since last reset.
>>>>>   * @return
>>>>>   */
>>>>>  public Class[] getEntries(){
>>>>>      return entrys.toArray(new Class[entrys.size()]);
>>>>>  }
>>>>>
>>>>>  @Override
>>>>>  public int hashCode() {
>>>>>      int hash = 3;
>>>>>      hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>>>> 0);
>>>>>      return hash;
>>>>>  }
>>>>>
>>>>>  @Override
>>>>>  public boolean equals(Object o){
>>>>>      if (o == null) return false;
>>>>>      if (o instanceof DefaultEntries){
>>>>>          if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>>>      }
>>>>>      return false;
>>>>>  }
>>>>>    @Override
>>>>>  public String toString(){
>>>>>      String newline = System.getProperty("line.separator");
>>>>>      StringBuilder sb = new StringBuilder(256);
>>>>>      sb.append("DefaultEntries:");
>>>>>      sb.append(newline);
>>>>>      Iterator<Class>  it = entrys.iterator();
>>>>>      while (it.hasNext()){
>>>>>          sb.append(it.next().getName());
>>>>>          sb.append(newline);
>>>>>      }
>>>>>      return sb.toString();
>>>>>  }
>>>>> }
>>>>>
>>>>>
>>>>>           
>
>   

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@gmail.com>]

Yes definitely the case.  CHM just makes it a lot easier to not worry about map and set data structures in terms of concurrent access of mutating data. Set and HashSet are useful, but not always in concurrent mutating data management.

I still prefer Vector over ArrayList just because it eliminates iteration problems that alway require the same lines of code to resolve.

Gregg

Sent from my iPhone

On Feb 7, 2011, at 2:08 PM, Dan Creswell <da...@gmail.com> wrote:

> All true, still need to document the concurrent contract for this class and
> implement accordingly.
> 
> i.e. We can say "toString" is thread safe etc and then use ConcurrentHashMap
> and friends.
> 
> But we gotta do it right....
> 
> On 7 February 2011 19:57, Gregg Wonderly <gr...@wonderly.org> wrote:
> 
>> On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>> 
>>> Sorry, I've got my nit-picking hat on...
>>> 
>>> The null check in the equals method isn't needed because you get a
>>> free one by using "instanceof".
>>> 
>>> I know you've warned that the class is not thread-safe, but people
>>> often forget that doing a sysout of an object calls toString().  I'd
>>> still be tempted to whack some protection into toString to stop it
>>> blowing up when people start printing lots of debug messages.
>>> 
>>> Other than that it looks good to me.
>>> 
>> 
>> I'd suggest that it would be easy enough to just use ConcurrentHashMap
>> instead of HashSet if we are going to move to Java 5 anyway.  Then, you
>> don't have to worry about concurrency issues with toString() etc.
>> 
>> Gregg Wonderly
>> 
>> 
>> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>> wrote:
>>> 
>>>> Dan Creswell wrote:
>>>> 
>>>>> 
>>>>> Solid - probably ought to be a constructor that doesn't add defaults
>>>>> too.....and maybe a note about thread safety on add.
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>> The finished article.
>>>> 
>>>> /*
>>>> * Licensed to the Apache Software Foundation (ASF) under one
>>>> * or more contributor license agreements.  See the NOTICE file
>>>> * distributed with this work for additional information
>>>> * regarding copyright ownership. The ASF licenses this file
>>>> * to you under the Apache License, Version 2.0 (the
>>>> * "License"); you may not use this file except in compliance
>>>> * with the License. You may obtain a copy of the License at
>>>> *
>>>> *      http://www.apache.org/licenses/LICENSE-2.0
>>>> *
>>>> * Unless required by applicable law or agreed to in writing, software
>>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>> implied.
>>>> * See the License for the specific language governing permissions and
>>>> * limitations under the License.
>>>> */
>>>> 
>>>> package org.apache.river.api.lookup;
>>>> 
>>>> import java.util.HashSet;
>>>> import java.util.Iterator;
>>>> import java.util.Set;
>>>> import net.jini.lookup.entry.Address;
>>>> import net.jini.lookup.entry.Comment;
>>>> import net.jini.lookup.entry.Location;
>>>> import net.jini.lookup.entry.Name;
>>>> import net.jini.lookup.entry.ServiceInfo;
>>>> import net.jini.lookup.entry.Status;
>>>> import net.jini.lookup.entry.UIDescriptor;
>>>> 
>>>> /**
>>>> * A little builder utility class that creates an array of Entry classes
>>>> to
>>>> * be used as a parameter for StreamServiceRegistrar.  All the jini
>>>> platform
>>>> * Entry's are included by default.
>>>> *
>>>> * Note: This class is not threadsafe, use external synchronization if
>>>> required.
>>>> *
>>>> * Suggested by Dan Creswell.
>>>> * @author peter
>>>> */
>>>> public class DefaultEntries {
>>>>  private final Set<Class>  entrys;
>>>>    public DefaultEntries() {
>>>>      entrys = new HashSet<Class>(16);
>>>>  }
>>>>  /**
>>>>   * Add an Entry class.
>>>>   * @param cl - class
>>>>   * @return this
>>>>   */
>>>>  public DefaultEntries add(Class cl){
>>>>      entrys.add(cl);
>>>>      return this;
>>>>  }
>>>>  /**
>>>>   * All all the Jini Platform Entry's
>>>>   * @return
>>>>   */
>>>>  public DefaultEntries addPlatformEntries(){
>>>>      add(Comment.class);
>>>>      add(Location.class);
>>>>      add(Name.class);
>>>>      add(ServiceInfo.class);
>>>>      add(Status.class);
>>>>      add(UIDescriptor.class);
>>>>      add(Address.class);
>>>>      return this;
>>>>  }
>>>>  /**
>>>>   * Remove all Entry's
>>>>   */
>>>>  public void reset(){
>>>>      entrys.clear();
>>>>  }
>>>>    /**
>>>>   * Generate a new array containing all Entry's added since last reset.
>>>>   * @return
>>>>   */
>>>>  public Class[] getEntries(){
>>>>      return entrys.toArray(new Class[entrys.size()]);
>>>>  }
>>>> 
>>>>  @Override
>>>>  public int hashCode() {
>>>>      int hash = 3;
>>>>      hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>>> 0);
>>>>      return hash;
>>>>  }
>>>> 
>>>>  @Override
>>>>  public boolean equals(Object o){
>>>>      if (o == null) return false;
>>>>      if (o instanceof DefaultEntries){
>>>>          if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>>      }
>>>>      return false;
>>>>  }
>>>>    @Override
>>>>  public String toString(){
>>>>      String newline = System.getProperty("line.separator");
>>>>      StringBuilder sb = new StringBuilder(256);
>>>>      sb.append("DefaultEntries:");
>>>>      sb.append(newline);
>>>>      Iterator<Class>  it = entrys.iterator();
>>>>      while (it.hasNext()){
>>>>          sb.append(it.next().getName());
>>>>          sb.append(newline);
>>>>      }
>>>>      return sb.toString();
>>>>  }
>>>> }
>>>> 
>>>> 
>>> 
>> 

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

All true, still need to document the concurrent contract for this class and
implement accordingly.

i.e. We can say "toString" is thread safe etc and then use ConcurrentHashMap
and friends.

But we gotta do it right....

On 7 February 2011 19:57, Gregg Wonderly <gr...@wonderly.org> wrote:

> On 2/7/2011 5:24 AM, Tom Hobbs wrote:
>
>> Sorry, I've got my nit-picking hat on...
>>
>> The null check in the equals method isn't needed because you get a
>> free one by using "instanceof".
>>
>> I know you've warned that the class is not thread-safe, but people
>> often forget that doing a sysout of an object calls toString().  I'd
>> still be tempted to whack some protection into toString to stop it
>> blowing up when people start printing lots of debug messages.
>>
>> Other than that it looks good to me.
>>
>
> I'd suggest that it would be easy enough to just use ConcurrentHashMap
> instead of HashSet if we are going to move to Java 5 anyway.  Then, you
> don't have to worry about concurrency issues with toString() etc.
>
> Gregg Wonderly
>
>
>  On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>
>>  wrote:
>>
>>> Dan Creswell wrote:
>>>
>>>>
>>>> Solid - probably ought to be a constructor that doesn't add defaults
>>>> too.....and maybe a note about thread safety on add.
>>>>
>>>>
>>>>
>>>>
>>> The finished article.
>>>
>>> /*
>>> * Licensed to the Apache Software Foundation (ASF) under one
>>> * or more contributor license agreements.  See the NOTICE file
>>> * distributed with this work for additional information
>>> * regarding copyright ownership. The ASF licenses this file
>>> * to you under the Apache License, Version 2.0 (the
>>> * "License"); you may not use this file except in compliance
>>> * with the License. You may obtain a copy of the License at
>>> *
>>> *      http://www.apache.org/licenses/LICENSE-2.0
>>> *
>>> * Unless required by applicable law or agreed to in writing, software
>>> * distributed under the License is distributed on an "AS IS" BASIS,
>>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>> implied.
>>> * See the License for the specific language governing permissions and
>>> * limitations under the License.
>>> */
>>>
>>> package org.apache.river.api.lookup;
>>>
>>> import java.util.HashSet;
>>> import java.util.Iterator;
>>> import java.util.Set;
>>> import net.jini.lookup.entry.Address;
>>> import net.jini.lookup.entry.Comment;
>>> import net.jini.lookup.entry.Location;
>>> import net.jini.lookup.entry.Name;
>>> import net.jini.lookup.entry.ServiceInfo;
>>> import net.jini.lookup.entry.Status;
>>> import net.jini.lookup.entry.UIDescriptor;
>>>
>>> /**
>>> * A little builder utility class that creates an array of Entry classes
>>> to
>>> * be used as a parameter for StreamServiceRegistrar.  All the jini
>>> platform
>>> * Entry's are included by default.
>>> *
>>> * Note: This class is not threadsafe, use external synchronization if
>>> required.
>>> *
>>> * Suggested by Dan Creswell.
>>> * @author peter
>>> */
>>> public class DefaultEntries {
>>>   private final Set<Class>  entrys;
>>>     public DefaultEntries() {
>>>       entrys = new HashSet<Class>(16);
>>>   }
>>>   /**
>>>    * Add an Entry class.
>>>    * @param cl - class
>>>    * @return this
>>>    */
>>>   public DefaultEntries add(Class cl){
>>>       entrys.add(cl);
>>>       return this;
>>>   }
>>>   /**
>>>    * All all the Jini Platform Entry's
>>>    * @return
>>>    */
>>>   public DefaultEntries addPlatformEntries(){
>>>       add(Comment.class);
>>>       add(Location.class);
>>>       add(Name.class);
>>>       add(ServiceInfo.class);
>>>       add(Status.class);
>>>       add(UIDescriptor.class);
>>>       add(Address.class);
>>>       return this;
>>>   }
>>>   /**
>>>    * Remove all Entry's
>>>    */
>>>   public void reset(){
>>>       entrys.clear();
>>>   }
>>>     /**
>>>    * Generate a new array containing all Entry's added since last reset.
>>>    * @return
>>>    */
>>>   public Class[] getEntries(){
>>>       return entrys.toArray(new Class[entrys.size()]);
>>>   }
>>>
>>>   @Override
>>>   public int hashCode() {
>>>       int hash = 3;
>>>       hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() :
>>> 0);
>>>       return hash;
>>>   }
>>>
>>>   @Override
>>>   public boolean equals(Object o){
>>>       if (o == null) return false;
>>>       if (o instanceof DefaultEntries){
>>>           if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>>       }
>>>       return false;
>>>   }
>>>     @Override
>>>   public String toString(){
>>>       String newline = System.getProperty("line.separator");
>>>       StringBuilder sb = new StringBuilder(256);
>>>       sb.append("DefaultEntries:");
>>>       sb.append(newline);
>>>       Iterator<Class>  it = entrys.iterator();
>>>       while (it.hasNext()){
>>>           sb.append(it.next().getName());
>>>           sb.append(newline);
>>>       }
>>>       return sb.toString();
>>>   }
>>> }
>>>
>>>
>>
>

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

On 2/7/2011 5:24 AM, Tom Hobbs wrote:
> Sorry, I've got my nit-picking hat on...
>
> The null check in the equals method isn't needed because you get a
> free one by using "instanceof".
>
> I know you've warned that the class is not thread-safe, but people
> often forget that doing a sysout of an object calls toString().  I'd
> still be tempted to whack some protection into toString to stop it
> blowing up when people start printing lots of debug messages.
>
> Other than that it looks good to me.

I'd suggest that it would be easy enough to just use ConcurrentHashMap instead 
of HashSet if we are going to move to Java 5 anyway.  Then, you don't have to 
worry about concurrency issues with toString() etc.

Gregg Wonderly

> On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone<ji...@zeus.net.au>  wrote:
>> Dan Creswell wrote:
>>>
>>> Solid - probably ought to be a constructor that doesn't add defaults
>>> too.....and maybe a note about thread safety on add.
>>>
>>>
>>>
>>
>> The finished article.
>>
>> /*
>> * Licensed to the Apache Software Foundation (ASF) under one
>> * or more contributor license agreements.  See the NOTICE file
>> * distributed with this work for additional information
>> * regarding copyright ownership. The ASF licenses this file
>> * to you under the Apache License, Version 2.0 (the
>> * "License"); you may not use this file except in compliance
>> * with the License. You may obtain a copy of the License at
>> *
>> *      http://www.apache.org/licenses/LICENSE-2.0
>> *
>> * Unless required by applicable law or agreed to in writing, software
>> * distributed under the License is distributed on an "AS IS" BASIS,
>> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> * See the License for the specific language governing permissions and
>> * limitations under the License.
>> */
>>
>> package org.apache.river.api.lookup;
>>
>> import java.util.HashSet;
>> import java.util.Iterator;
>> import java.util.Set;
>> import net.jini.lookup.entry.Address;
>> import net.jini.lookup.entry.Comment;
>> import net.jini.lookup.entry.Location;
>> import net.jini.lookup.entry.Name;
>> import net.jini.lookup.entry.ServiceInfo;
>> import net.jini.lookup.entry.Status;
>> import net.jini.lookup.entry.UIDescriptor;
>>
>> /**
>> * A little builder utility class that creates an array of Entry classes to
>> * be used as a parameter for StreamServiceRegistrar.  All the jini platform
>> * Entry's are included by default.
>> *
>> * Note: This class is not threadsafe, use external synchronization if
>> required.
>> *
>> * Suggested by Dan Creswell.
>> * @author peter
>> */
>> public class DefaultEntries {
>>    private final Set<Class>  entrys;
>>      public DefaultEntries() {
>>        entrys = new HashSet<Class>(16);
>>    }
>>    /**
>>     * Add an Entry class.
>>     * @param cl - class
>>     * @return this
>>     */
>>    public DefaultEntries add(Class cl){
>>        entrys.add(cl);
>>        return this;
>>    }
>>    /**
>>     * All all the Jini Platform Entry's
>>     * @return
>>     */
>>    public DefaultEntries addPlatformEntries(){
>>        add(Comment.class);
>>        add(Location.class);
>>        add(Name.class);
>>        add(ServiceInfo.class);
>>        add(Status.class);
>>        add(UIDescriptor.class);
>>        add(Address.class);
>>        return this;
>>    }
>>    /**
>>     * Remove all Entry's
>>     */
>>    public void reset(){
>>        entrys.clear();
>>    }
>>      /**
>>     * Generate a new array containing all Entry's added since last reset.
>>     * @return
>>     */
>>    public Class[] getEntries(){
>>        return entrys.toArray(new Class[entrys.size()]);
>>    }
>>
>>    @Override
>>    public int hashCode() {
>>        int hash = 3;
>>        hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() : 0);
>>        return hash;
>>    }
>>
>>    @Override
>>    public boolean equals(Object o){
>>        if (o == null) return false;
>>        if (o instanceof DefaultEntries){
>>            if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>>        }
>>        return false;
>>    }
>>      @Override
>>    public String toString(){
>>        String newline = System.getProperty("line.separator");
>>        StringBuilder sb = new StringBuilder(256);
>>        sb.append("DefaultEntries:");
>>        sb.append(newline);
>>        Iterator<Class>  it = entrys.iterator();
>>        while (it.hasNext()){
>>            sb.append(it.next().getName());
>>            sb.append(newline);
>>        }
>>        return sb.toString();
>>    }
>> }
>>
>

Re: MarshalledServiceItem

[Tom Hobbs <tv...@googlemail.com>]

Sorry, I've got my nit-picking hat on...

The null check in the equals method isn't needed because you get a
free one by using "instanceof".

I know you've warned that the class is not thread-safe, but people
often forget that doing a sysout of an object calls toString().  I'd
still be tempted to whack some protection into toString to stop it
blowing up when people start printing lots of debug messages.

Other than that it looks good to me.

On Mon, Feb 7, 2011 at 10:56 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> Dan Creswell wrote:
>>
>> Solid - probably ought to be a constructor that doesn't add defaults
>> too.....and maybe a note about thread safety on add.
>>
>>
>>
>
> The finished article.
>
> /*
> * Licensed to the Apache Software Foundation (ASF) under one
> * or more contributor license agreements.  See the NOTICE file
> * distributed with this work for additional information
> * regarding copyright ownership. The ASF licenses this file
> * to you under the Apache License, Version 2.0 (the
> * "License"); you may not use this file except in compliance
> * with the License. You may obtain a copy of the License at
> *
> *      http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
>
> package org.apache.river.api.lookup;
>
> import java.util.HashSet;
> import java.util.Iterator;
> import java.util.Set;
> import net.jini.lookup.entry.Address;
> import net.jini.lookup.entry.Comment;
> import net.jini.lookup.entry.Location;
> import net.jini.lookup.entry.Name;
> import net.jini.lookup.entry.ServiceInfo;
> import net.jini.lookup.entry.Status;
> import net.jini.lookup.entry.UIDescriptor;
>
> /**
> * A little builder utility class that creates an array of Entry classes to
> * be used as a parameter for StreamServiceRegistrar.  All the jini platform
> * Entry's are included by default.
> *
> * Note: This class is not threadsafe, use external synchronization if
> required.
> *
> * Suggested by Dan Creswell.
> * @author peter
> */
> public class DefaultEntries {
>   private final Set<Class> entrys;
>     public DefaultEntries() {
>       entrys = new HashSet<Class>(16);
>   }
>   /**
>    * Add an Entry class.
>    * @param cl - class
>    * @return this
>    */
>   public DefaultEntries add(Class cl){
>       entrys.add(cl);
>       return this;
>   }
>   /**
>    * All all the Jini Platform Entry's
>    * @return
>    */
>   public DefaultEntries addPlatformEntries(){
>       add(Comment.class);
>       add(Location.class);
>       add(Name.class);
>       add(ServiceInfo.class);
>       add(Status.class);
>       add(UIDescriptor.class);
>       add(Address.class);
>       return this;
>   }
>   /**
>    * Remove all Entry's
>    */
>   public void reset(){
>       entrys.clear();
>   }
>     /**
>    * Generate a new array containing all Entry's added since last reset.
>    * @return
>    */
>   public Class[] getEntries(){
>       return entrys.toArray(new Class[entrys.size()]);
>   }
>
>   @Override
>   public int hashCode() {
>       int hash = 3;
>       hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() : 0);
>       return hash;
>   }
>
>   @Override
>   public boolean equals(Object o){
>       if (o == null) return false;
>       if (o instanceof DefaultEntries){
>           if (entrys.equals(((DefaultEntries)o).entrys)) return true;
>       }
>       return false;
>   }
>     @Override
>   public String toString(){
>       String newline = System.getProperty("line.separator");
>       StringBuilder sb = new StringBuilder(256);
>       sb.append("DefaultEntries:");
>       sb.append(newline);
>       Iterator<Class> it = entrys.iterator();
>       while (it.hasNext()){
>           sb.append(it.next().getName());
>           sb.append(newline);
>       }
>       return sb.toString();
>   }
> }
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Dan Creswell wrote:
> Solid - probably ought to be a constructor that doesn't add defaults
> too.....and maybe a note about thread safety on add.
>
>
>   
The finished article.

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.river.api.lookup;

import java.util.HashSet;
import java.util.Iterator;
import java.util.Set;
import net.jini.lookup.entry.Address;
import net.jini.lookup.entry.Comment;
import net.jini.lookup.entry.Location;
import net.jini.lookup.entry.Name;
import net.jini.lookup.entry.ServiceInfo;
import net.jini.lookup.entry.Status;
import net.jini.lookup.entry.UIDescriptor;

/**
 * A little builder utility class that creates an array of Entry classes to
 * be used as a parameter for StreamServiceRegistrar.  All the jini platform
 * Entry's are included by default.
 *
 * Note: This class is not threadsafe, use external synchronization if 
required.
 *
 * Suggested by Dan Creswell.
 * @author peter
 */
public class DefaultEntries {
    private final Set<Class> entrys;
   
    public DefaultEntries() {
        entrys = new HashSet<Class>(16);
    }
    /**
     * Add an Entry class.
     * @param cl - class
     * @return this
     */
    public DefaultEntries add(Class cl){
        entrys.add(cl);
        return this;
    }
    /**
     * All all the Jini Platform Entry's
     * @return
     */
    public DefaultEntries addPlatformEntries(){
        add(Comment.class);
        add(Location.class);
        add(Name.class);
        add(ServiceInfo.class);
        add(Status.class);
        add(UIDescriptor.class);
        add(Address.class);
        return this;
    }
    /**
     * Remove all Entry's
     */
    public void reset(){
        entrys.clear();
    }
   
    /**
     * Generate a new array containing all Entry's added since last reset.
     * @return
     */
    public Class[] getEntries(){
        return entrys.toArray(new Class[entrys.size()]);
    }

    @Override
    public int hashCode() {
        int hash = 3;
        hash = 29 * hash + (this.entrys != null ? this.entrys.hashCode() 
: 0);
        return hash;
    }

    @Override
    public boolean equals(Object o){
        if (o == null) return false;
        if (o instanceof DefaultEntries){
            if (entrys.equals(((DefaultEntries)o).entrys)) return true;
        }
        return false;
    }
   
    @Override
    public String toString(){
        String newline = System.getProperty("line.separator");
        StringBuilder sb = new StringBuilder(256);
        sb.append("DefaultEntries:");
        sb.append(newline);
        Iterator<Class> it = entrys.iterator();
        while (it.hasNext()){
            sb.append(it.next().getName());
            sb.append(newline);
        }
        return sb.toString();
    }
}

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

Solid - probably ought to be a constructor that doesn't add defaults
too.....and maybe a note about thread safety on add.

On 7 February 2011 04:29, Peter Firmstone <ji...@zeus.net.au> wrote:

> Dan Creswell wrote:
>
>>
>>  Sorry, what you meant was the builder generates a default array or Set of
>>> Entry classes required to support the platform, additional Entry classes
>>> the
>>> client needs can be added, then it builds the array to pass as a
>>> parameter
>>> to the lookup service.
>>>
>>>
>>>
>>>
>> Yep.
>>
>>
>>
>>
>
> How's this?
>
> Cheers,
>
> Peter.
>
>
> /*
> * Licensed to the Apache Software Foundation (ASF) under one
> * or more contributor license agreements.  See the NOTICE file
> * distributed with this work for additional information
> * regarding copyright ownership. The ASF licenses this file
> * to you under the Apache License, Version 2.0 (the
> * "License"); you may not use this file except in compliance
> * with the License. You may obtain a copy of the License at
> *
> *      http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
>
> package org.apache.river.api.lookup;
>
> import java.util.HashSet;
> import java.util.Set;
> import net.jini.lookup.entry.Address;
> import net.jini.lookup.entry.Comment;
> import net.jini.lookup.entry.Location;
> import net.jini.lookup.entry.Name;
> import net.jini.lookup.entry.ServiceInfo;
> import net.jini.lookup.entry.Status;
> import net.jini.lookup.entry.UIDescriptor;
>
> /**
> * A little builder utility class that creates an array of Entry classes to
> * be used as a parameter for StreamServiceRegistrar.  All the platform
> * Entry's are included.
> *
> * Suggested by Dan Creswell.
> * @author peter
> */
> public class DefaultEntries {
>   private final Set<Class> entrys;
>   public DefaultEntries() {
>       entrys = new HashSet<Class>(16);
>       add(Comment.class);
>       add(Location.class);
>       add(Name.class);
>       add(ServiceInfo.class);
>       add(Status.class);
>       add(UIDescriptor.class);
>       add(Address.class);
>   }
>   public DefaultEntries add(Class cl){
>       entrys.add(cl);
>       return this;
>   }
>   public Class[] getEntries(){
>       return entrys.toArray(new Class[entrys.size()]);
>   }
>
> }
>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Dan Creswell wrote:
>
>> Sorry, what you meant was the builder generates a default array or Set of
>> Entry classes required to support the platform, additional Entry classes the
>> client needs can be added, then it builds the array to pass as a parameter
>> to the lookup service.
>>
>>
>>     
> Yep.
>
>
>   

How's this?

Cheers,

Peter.

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.river.api.lookup;

import java.util.HashSet;
import java.util.Set;
import net.jini.lookup.entry.Address;
import net.jini.lookup.entry.Comment;
import net.jini.lookup.entry.Location;
import net.jini.lookup.entry.Name;
import net.jini.lookup.entry.ServiceInfo;
import net.jini.lookup.entry.Status;
import net.jini.lookup.entry.UIDescriptor;

/**
 * A little builder utility class that creates an array of Entry classes to
 * be used as a parameter for StreamServiceRegistrar.  All the platform
 * Entry's are included.
 *
 * Suggested by Dan Creswell.
 * @author peter
 */
public class DefaultEntries {
    private final Set<Class> entrys;
    public DefaultEntries() {
        entrys = new HashSet<Class>(16);
        add(Comment.class);
        add(Location.class);
        add(Name.class);
        add(ServiceInfo.class);
        add(Status.class);
        add(UIDescriptor.class);
        add(Address.class);
    }
    public DefaultEntries add(Class cl){
        entrys.add(cl);
        return this;
    }
    public Class[] getEntries(){
        return entrys.toArray(new Class[entrys.size()]);
    }

}

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

S'cool.

On 6 February 2011 19:45, Peter Firmstone <ji...@zeus.net.au> wrote:

> I was thinking about post filtering on an Entry's contents after the lookup
> method returns.
>
>
Well, there might be a use case there as well, not sure but than haven't
thought about it....


> Sorry, what you meant was the builder generates a default array or Set of
> Entry classes required to support the platform, additional Entry classes the
> client needs can be added, then it builds the array to pass as a parameter
> to the lookup service.
>
>
Yep.


> Do you think we need to pass all classes related to the Entry's (fields) or
> just the Entry classes?  I'm thinking it'd be simpler if it's just the Entry
> classes.
>
>
I'm thinking classes, anything more than that is more specific selection
that a client wants to do and hasn't accounted for in the original lookup
(which would seem to be a "broken" client?)


>
> Cheers,
>
> Peter.
>
> Dan Creswell wrote:
>
>> I was assuming a filter was inclusive by default. i.e. You specify just
>> the
>> things you're interested in a client seeing.
>>
>> There seems less value in removal given a client specifies its interests
>> somewhat in relation to what is on its classpath....
>>
>> On 6 February 2011 11:45, Peter Firmstone <ji...@zeus.net.au> wrote:
>>
>>
>>
>>> A filter could also collect, rather than just remove.
>>>
>>> Do you think query/filter builders should be specific to a particular
>>> Entry
>>> type?
>>>
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>> Dan Creswell wrote:
>>>
>>>
>>>
>>>> Yep.
>>>>
>>>> On 5 February 2011 10:45, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Sounds interesting, looks like a builder?
>>>>>
>>>>>
>>>>> Dan Creswell wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I'm also figuring you might have some wise defaults that people could
>>>>>> add
>>>>>> to
>>>>>> for constructing a filter/matchset.
>>>>>>
>>>>>> Something like:
>>>>>>
>>>>>> DefaultEntrys.add(...).add(....)
>>>>>>
>>>>>> DefaultEntrys probably includes something like ServiceInfo, Name and
>>>>>> such
>>>>>> -
>>>>>> all stock options in the platform...
>>>>>>
>>>>>> On 5 February 2011 10:28, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>>>>>>>
>>>>>>> So we make the Entry's the client's interested in, the only Entry's
>>>>>>> that'll
>>>>>>> be unmarshalled, those and the ServiceID.
>>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Peter.
>>>>>>>
>>>>>>> Dan Creswell wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> ...
>>>>>>>>
>>>>>>>> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Dan Creswell wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> "ServiceClasspathSubItem is intended for client side filtering of
>>>>>>>>>> lookup
>>>>>>>>>> * service results prior to clients using a service, the lookup
>>>>>>>>>> service
>>>>>>>>>> * that implements this class, implements #getServiceItem(), so
>>>>>>>>>> clients
>>>>>>>>>> * can obtain a complete ServiceItem when required after
>>>>>>>>>> filtering."
>>>>>>>>>>
>>>>>>>>>> I'm still wondering if it wouldn't be better for a client to send
>>>>>>>>>> a
>>>>>>>>>> list
>>>>>>>>>> of
>>>>>>>>>> Entry's it's interested in seeing, separate from the set for
>>>>>>>>>> matching
>>>>>>>>>> to
>>>>>>>>>> the
>>>>>>>>>> Registrar. The Registrar can then return ServiceItems for the
>>>>>>>>>> matches
>>>>>>>>>> and
>>>>>>>>>> pre-filter the Entry's per item. Especially given:
>>>>>>>>>>
>>>>>>>>>> "Some fields in ServiceClasspathSubItem may be null, fields in
>>>>>>>>>> Entry's
>>>>>>>>>> may
>>>>>>>>>> * be null or even the service reference may be null, these fields
>>>>>>>>>> would
>>>>>>>>>> be
>>>>>>>>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>>>>>>>>> downloaded
>>>>>>>>>> * code or a remote codebase."
>>>>>>>>>>
>>>>>>>>>> I think, if we do the filtering as I suggest, much of the comment
>>>>>>>>>> re:
>>>>>>>>>> nulls
>>>>>>>>>> above still applies in implementation but ultimately, the client
>>>>>>>>>> will
>>>>>>>>>> pick
>>>>>>>>>> Entry's for which it has the classpath such that it always deals
>>>>>>>>>> in
>>>>>>>>>> complete
>>>>>>>>>> Entry's and can have a level of predictability in respect of what
>>>>>>>>>> is
>>>>>>>>>> or
>>>>>>>>>> is
>>>>>>>>>> not null. This might save clients having to do endless null checks
>>>>>>>>>> etc
>>>>>>>>>> that
>>>>>>>>>> can be a rich source of bugs and ugly code.
>>>>>>>>>>
>>>>>>>>>> We might need the filter list to include specific subclasses as
>>>>>>>>>> well....
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Hmm, null is a problem, we could still have fields missing subclass
>>>>>>>>> classes, which might prevent a subclass from unmarshalling.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Yeah, which is why I'm tempted to have the client be explicit about
>>>>>>>> which
>>>>>>>> Entry's (and that would include subclasses of those Entry's it can
>>>>>>>> tolerate). In this way the client can provide high-level explicit
>>>>>>>> guidance
>>>>>>>> as to what's right. No need for code to try and be clever and guess
>>>>>>>> the
>>>>>>>> client intent which is always ugly.
>>>>>>>>
>>>>>>>> Many times I suspect the Entry set used for matching will be the
>>>>>>>> same
>>>>>>>> as
>>>>>>>> the
>>>>>>>> filter set, probably can provide a nice method signature to help
>>>>>>>> that.
>>>>>>>>
>>>>>>>> Is there a way we can catch a class not found exception or similar
>>>>>>>> and
>>>>>>>> still
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> get the state of the superclass, or would this violate
>>>>>>>>> polymorphism,
>>>>>>>>> method
>>>>>>>>> overrides etc?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Probably but again this is building clever code to try and
>>>>>>>> compensate.
>>>>>>>> It's
>>>>>>>> the same as writing all the null handling if you will albeit hidden
>>>>>>>> inside
>>>>>>>> the River libraries.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I guess the likelihood of having null values is reduced, although
>>>>>>>>> not
>>>>>>>>> eliminated.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Quite.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> If we compact the final array of Entry's so null entry's are
>>>>>>>>> removed,
>>>>>>>>> the
>>>>>>>>> client only needs to find the Entry's it's interested in, likely
>>>>>>>>> ignoring
>>>>>>>>> the others anyway.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Sure but then the client has to explicitly construct filter code,
>>>>>>>> albeit
>>>>>>>> clean of null checks to go through the list of Entry's to find what
>>>>>>>> it's
>>>>>>>> interested in. Unless we return a HashMap(EntryClass.getName,
>>>>>>>> EntryClass.instance) to make that easier. Even so, you're still
>>>>>>>> returning
>>>>>>>> a
>>>>>>>> bunch of Entry's only some of which may be interesting to the
>>>>>>>> client....
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> So the client needs to ensure the Entry's are resolvable from the
>>>>>>>>> classpath.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Absolutely - and the fact that it must means it can thus construct a
>>>>>>>> proper
>>>>>>>> filter list of Entrys of interest.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I have thought about the case where a proxy uses the lookup service
>>>>>>>>> and
>>>>>>>>> has
>>>>>>>>> Entry's in it's proxy classloader, but this could be subject to
>>>>>>>>> security
>>>>>>>>> issues, where two services with different Subject's end up sharing
>>>>>>>>> state.
>>>>>>>>>
>>>>>>>>> We might also consider only including Entry's that don't have null
>>>>>>>>> field
>>>>>>>>> values, this puts the impetus on the implementer to consider the
>>>>>>>>> Entry
>>>>>>>>> field
>>>>>>>>> classes.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Again, I think we're venturing into attempting to imply intent by
>>>>>>>> examining
>>>>>>>> null fields. If the implementer makes a mistake in their analysis
>>>>>>>> they
>>>>>>>> end
>>>>>>>> up with unexpected results come lookup. Personally I don't like this
>>>>>>>> sort
>>>>>>>> of
>>>>>>>> thing, I like explicit methods to guide an implementer and eliminate
>>>>>>>> opportunities for these kinds of confusion.
>>>>>>>>
>>>>>>>> That last bit is really why I'm down on trying to solve all these
>>>>>>>> problems
>>>>>>>> (including the bandwidth ones) with clever management of
>>>>>>>> class-downloading,
>>>>>>>> reflection and such. It's implicit, makes assumptions about usage
>>>>>>>> that
>>>>>>>> are
>>>>>>>> difficult for users to account for. It also leads to complicated,
>>>>>>>> brittle
>>>>>>>> code with many an opportunity for dark-corners/unhandled edge cases
>>>>>>>> that
>>>>>>>> cause confusion and are difficult to identify and fix for River
>>>>>>>> implementers
>>>>>>>> and users.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Peter.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

One thing I wanted to do is document the list of classnames that I found 
necessary to "neverPrefer" in order to keep PreferredClassLoader from 
downloading anything during Entry unmarshalling.  There are some of my own 
classnames in this list, but also there are lots of different low level class 
names which all get resolved as part of the ClassLoading triggered by 
PreferredClassLoader using the codebase's class loader (usually set as the 
Thread's contextClassLoader).

		neverPrefer( Name.class.getName() );
		neverPrefer( ServiceInfo.class.getName() );
		neverPrefer( ServiceType.class.getName() );
		neverPrefer( ConsistentMap.class.getName() );
		neverPrefer( DesktopIcon.class.getName() );
		neverPrefer( DesktopCodebase.class.getName() );
		neverPrefer( DesktopEntry.class.getName() );
		neverPrefer( DesktopGroup.class.getName() );
		neverPrefer( URL.class.getName() );
		neverPrefer( URI.class.getName() );
		neverPrefer( UIDescriptor.class.getName() );
		neverPrefer( "org.wonderly.jini2.browse.ApplicationUIDescriptor" );
		neverPrefer( "com.cytetech.jini.ApplicationUIDescriptor" );
		neverPrefer( AdminDescriptor.class.getName() );
		neverPrefer( JavaHelpEntry.class.getName() );
		neverPrefer( HtmlHelpEntry.class.getName() );
		neverPrefer( HelpEntry.class.getName() );
		neverPrefer( MarshalledObject.class.getName() );
		neverPrefer( java.lang.reflect.Proxy.class.getName() );
		neverPrefer( net.jini.jeri.BasicInvocationHandler.class.getName() );
		neverPrefer( net.jini.jeri.BasicObjectEndpoint.class.getName() );
		neverPrefer( net.jini.jeri.BasicInvocationHandler.class.getName() );
		neverPrefer( net.jini.jeri.ssl.SslEndpoint.class.getName() );
		neverPrefer( net.jini.jeri.tcp.TcpEndpoint.class.getName() );
		neverPrefer( net.jini.id.UuidFactory.class.getName() );
		neverPrefer( net.jini.id.Uuid.class.getName() );
		neverPrefer( com.artima.lookup.util.ConsistentSet.class.getName() );
		neverPrefer( Object[].class.getName() );
		neverPrefer( byte[].class.getName() );

If we keep the codebase ClassLoader out of view, then we can not really worry 
too much about this list.  But, I just wanted to make it clear to others, that 
don't understand what's happening under the covers, that there are considerable 
points of class resolution where the codebase's class loader can be accessed.

Gregg Wonderly

On 2/6/2011 1:45 PM, Peter Firmstone wrote:
> I was thinking about post filtering on an Entry's contents after the lookup
> method returns.
>
> Sorry, what you meant was the builder generates a default array or Set of Entry
> classes required to support the platform, additional Entry classes the client
> needs can be added, then it builds the array to pass as a parameter to the
> lookup service.
>
> Do you think we need to pass all classes related to the Entry's (fields) or just
> the Entry classes? I'm thinking it'd be simpler if it's just the Entry classes.
>
> Cheers,
>
> Peter.
>
> Dan Creswell wrote:
>> I was assuming a filter was inclusive by default. i.e. You specify just the
>> things you're interested in a client seeing.
>>
>> There seems less value in removal given a client specifies its interests
>> somewhat in relation to what is on its classpath....
>>
>> On 6 February 2011 11:45, Peter Firmstone <ji...@zeus.net.au> wrote:
>>
>>> A filter could also collect, rather than just remove.
>>>
>>> Do you think query/filter builders should be specific to a particular Entry
>>> type?
>>>
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>> Dan Creswell wrote:
>>>
>>>> Yep.
>>>>
>>>> On 5 February 2011 10:45, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>
>>>>
>>>>
>>>>> Sounds interesting, looks like a builder?
>>>>>
>>>>>
>>>>> Dan Creswell wrote:
>>>>>
>>>>>
>>>>>
>>>>>> I'm also figuring you might have some wise defaults that people could
>>>>>> add
>>>>>> to
>>>>>> for constructing a filter/matchset.
>>>>>>
>>>>>> Something like:
>>>>>>
>>>>>> DefaultEntrys.add(...).add(....)
>>>>>>
>>>>>> DefaultEntrys probably includes something like ServiceInfo, Name and
>>>>>> such
>>>>>> -
>>>>>> all stock options in the platform...
>>>>>>
>>>>>> On 5 February 2011 10:28, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>>>>>>>
>>>>>>> So we make the Entry's the client's interested in, the only Entry's
>>>>>>> that'll
>>>>>>> be unmarshalled, those and the ServiceID.
>>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Peter.
>>>>>>>
>>>>>>> Dan Creswell wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> ...
>>>>>>>>
>>>>>>>> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Dan Creswell wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> "ServiceClasspathSubItem is intended for client side filtering of
>>>>>>>>>> lookup
>>>>>>>>>> * service results prior to clients using a service, the lookup
>>>>>>>>>> service
>>>>>>>>>> * that implements this class, implements #getServiceItem(), so
>>>>>>>>>> clients
>>>>>>>>>> * can obtain a complete ServiceItem when required after filtering."
>>>>>>>>>>
>>>>>>>>>> I'm still wondering if it wouldn't be better for a client to send a
>>>>>>>>>> list
>>>>>>>>>> of
>>>>>>>>>> Entry's it's interested in seeing, separate from the set for
>>>>>>>>>> matching
>>>>>>>>>> to
>>>>>>>>>> the
>>>>>>>>>> Registrar. The Registrar can then return ServiceItems for the
>>>>>>>>>> matches
>>>>>>>>>> and
>>>>>>>>>> pre-filter the Entry's per item. Especially given:
>>>>>>>>>>
>>>>>>>>>> "Some fields in ServiceClasspathSubItem may be null, fields in
>>>>>>>>>> Entry's
>>>>>>>>>> may
>>>>>>>>>> * be null or even the service reference may be null, these fields
>>>>>>>>>> would
>>>>>>>>>> be
>>>>>>>>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>>>>>>>>> downloaded
>>>>>>>>>> * code or a remote codebase."
>>>>>>>>>>
>>>>>>>>>> I think, if we do the filtering as I suggest, much of the comment
>>>>>>>>>> re:
>>>>>>>>>> nulls
>>>>>>>>>> above still applies in implementation but ultimately, the client
>>>>>>>>>> will
>>>>>>>>>> pick
>>>>>>>>>> Entry's for which it has the classpath such that it always deals in
>>>>>>>>>> complete
>>>>>>>>>> Entry's and can have a level of predictability in respect of what is
>>>>>>>>>> or
>>>>>>>>>> is
>>>>>>>>>> not null. This might save clients having to do endless null checks
>>>>>>>>>> etc
>>>>>>>>>> that
>>>>>>>>>> can be a rich source of bugs and ugly code.
>>>>>>>>>>
>>>>>>>>>> We might need the filter list to include specific subclasses as
>>>>>>>>>> well....
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Hmm, null is a problem, we could still have fields missing subclass
>>>>>>>>> classes, which might prevent a subclass from unmarshalling.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Yeah, which is why I'm tempted to have the client be explicit about
>>>>>>>> which
>>>>>>>> Entry's (and that would include subclasses of those Entry's it can
>>>>>>>> tolerate). In this way the client can provide high-level explicit
>>>>>>>> guidance
>>>>>>>> as to what's right. No need for code to try and be clever and guess
>>>>>>>> the
>>>>>>>> client intent which is always ugly.
>>>>>>>>
>>>>>>>> Many times I suspect the Entry set used for matching will be the same
>>>>>>>> as
>>>>>>>> the
>>>>>>>> filter set, probably can provide a nice method signature to help that.
>>>>>>>>
>>>>>>>> Is there a way we can catch a class not found exception or similar and
>>>>>>>> still
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> get the state of the superclass, or would this violate polymorphism,
>>>>>>>>> method
>>>>>>>>> overrides etc?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Probably but again this is building clever code to try and compensate.
>>>>>>>> It's
>>>>>>>> the same as writing all the null handling if you will albeit hidden
>>>>>>>> inside
>>>>>>>> the River libraries.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I guess the likelihood of having null values is reduced, although not
>>>>>>>>> eliminated.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Quite.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> If we compact the final array of Entry's so null entry's are removed,
>>>>>>>>> the
>>>>>>>>> client only needs to find the Entry's it's interested in, likely
>>>>>>>>> ignoring
>>>>>>>>> the others anyway.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Sure but then the client has to explicitly construct filter code,
>>>>>>>> albeit
>>>>>>>> clean of null checks to go through the list of Entry's to find what
>>>>>>>> it's
>>>>>>>> interested in. Unless we return a HashMap(EntryClass.getName,
>>>>>>>> EntryClass.instance) to make that easier. Even so, you're still
>>>>>>>> returning
>>>>>>>> a
>>>>>>>> bunch of Entry's only some of which may be interesting to the
>>>>>>>> client....
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> So the client needs to ensure the Entry's are resolvable from the
>>>>>>>>> classpath.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Absolutely - and the fact that it must means it can thus construct a
>>>>>>>> proper
>>>>>>>> filter list of Entrys of interest.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I have thought about the case where a proxy uses the lookup service
>>>>>>>>> and
>>>>>>>>> has
>>>>>>>>> Entry's in it's proxy classloader, but this could be subject to
>>>>>>>>> security
>>>>>>>>> issues, where two services with different Subject's end up sharing
>>>>>>>>> state.
>>>>>>>>>
>>>>>>>>> We might also consider only including Entry's that don't have null
>>>>>>>>> field
>>>>>>>>> values, this puts the impetus on the implementer to consider the
>>>>>>>>> Entry
>>>>>>>>> field
>>>>>>>>> classes.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Again, I think we're venturing into attempting to imply intent by
>>>>>>>> examining
>>>>>>>> null fields. If the implementer makes a mistake in their analysis they
>>>>>>>> end
>>>>>>>> up with unexpected results come lookup. Personally I don't like this
>>>>>>>> sort
>>>>>>>> of
>>>>>>>> thing, I like explicit methods to guide an implementer and eliminate
>>>>>>>> opportunities for these kinds of confusion.
>>>>>>>>
>>>>>>>> That last bit is really why I'm down on trying to solve all these
>>>>>>>> problems
>>>>>>>> (including the bandwidth ones) with clever management of
>>>>>>>> class-downloading,
>>>>>>>> reflection and such. It's implicit, makes assumptions about usage that
>>>>>>>> are
>>>>>>>> difficult for users to account for. It also leads to complicated,
>>>>>>>> brittle
>>>>>>>> code with many an opportunity for dark-corners/unhandled edge cases
>>>>>>>> that
>>>>>>>> cause confusion and are difficult to identify and fix for River
>>>>>>>> implementers
>>>>>>>> and users.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Peter.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

I was thinking about post filtering on an Entry's contents after the 
lookup method returns.

Sorry, what you meant was the builder generates a default array or Set 
of Entry classes required to support the platform, additional Entry 
classes the client needs can be added, then it builds the array to pass 
as a parameter to the lookup service.

Do you think we need to pass all classes related to the Entry's (fields) 
or just the Entry classes?  I'm thinking it'd be simpler if it's just 
the Entry classes.

Cheers,

Peter.

Dan Creswell wrote:
> I was assuming a filter was inclusive by default. i.e. You specify just the
> things you're interested in a client seeing.
>
> There seems less value in removal given a client specifies its interests
> somewhat in relation to what is on its classpath....
>
> On 6 February 2011 11:45, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> A filter could also collect, rather than just remove.
>>
>> Do you think query/filter builders should be specific to a particular Entry
>> type?
>>
>>
>> Cheers,
>>
>> Peter.
>>
>> Dan Creswell wrote:
>>
>>     
>>> Yep.
>>>
>>> On 5 February 2011 10:45, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>
>>>
>>>
>>>       
>>>> Sounds interesting, looks like a builder?
>>>>
>>>>
>>>> Dan Creswell wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> I'm also figuring you might have some wise defaults that people could
>>>>> add
>>>>> to
>>>>> for constructing a filter/matchset.
>>>>>
>>>>> Something like:
>>>>>
>>>>> DefaultEntrys.add(...).add(....)
>>>>>
>>>>> DefaultEntrys probably includes something like ServiceInfo, Name and
>>>>> such
>>>>> -
>>>>> all stock options in the platform...
>>>>>
>>>>> On 5 February 2011 10:28, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>>>>>>
>>>>>> So we make the Entry's the client's interested in, the only Entry's
>>>>>> that'll
>>>>>> be unmarshalled, those and the ServiceID.
>>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Peter.
>>>>>>
>>>>>> Dan Creswell wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> ...
>>>>>>>
>>>>>>> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Dan Creswell wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> "ServiceClasspathSubItem is intended for client side filtering of
>>>>>>>>> lookup
>>>>>>>>> * service results prior to clients using a service, the lookup
>>>>>>>>> service
>>>>>>>>> * that implements this class, implements #getServiceItem(), so
>>>>>>>>> clients
>>>>>>>>> * can obtain a complete ServiceItem when required after filtering."
>>>>>>>>>
>>>>>>>>> I'm still wondering if it wouldn't be better for a client to send a
>>>>>>>>> list
>>>>>>>>> of
>>>>>>>>> Entry's it's interested in seeing, separate from the set for
>>>>>>>>> matching
>>>>>>>>> to
>>>>>>>>> the
>>>>>>>>> Registrar. The Registrar can then return ServiceItems for the
>>>>>>>>> matches
>>>>>>>>> and
>>>>>>>>> pre-filter the Entry's per item. Especially given:
>>>>>>>>>
>>>>>>>>> "Some fields in ServiceClasspathSubItem may be null, fields in
>>>>>>>>> Entry's
>>>>>>>>> may
>>>>>>>>> * be null or even the service reference may be null, these fields
>>>>>>>>> would
>>>>>>>>> be
>>>>>>>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>>>>>>>> downloaded
>>>>>>>>> * code or a remote codebase."
>>>>>>>>>
>>>>>>>>> I think, if we do the filtering as I suggest, much of the comment
>>>>>>>>> re:
>>>>>>>>> nulls
>>>>>>>>> above still applies in implementation but ultimately, the client
>>>>>>>>> will
>>>>>>>>> pick
>>>>>>>>> Entry's for which it has the classpath such that it always deals in
>>>>>>>>> complete
>>>>>>>>> Entry's and can have a level of predictability in respect of what is
>>>>>>>>> or
>>>>>>>>> is
>>>>>>>>> not null. This might save clients having to do endless null checks
>>>>>>>>> etc
>>>>>>>>> that
>>>>>>>>> can be a rich source of bugs and ugly code.
>>>>>>>>>
>>>>>>>>> We might need the filter list to include specific subclasses as
>>>>>>>>> well....
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> Hmm, null is a problem, we could still have fields missing subclass
>>>>>>>> classes, which might prevent a subclass from unmarshalling.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> Yeah, which is why I'm tempted to have the client be explicit about
>>>>>>> which
>>>>>>> Entry's (and that would include subclasses of those Entry's it can
>>>>>>> tolerate). In this way the client can provide high-level explicit
>>>>>>> guidance
>>>>>>> as to what's right. No need for code to try and be clever and guess
>>>>>>> the
>>>>>>> client intent which is always ugly.
>>>>>>>
>>>>>>> Many times I suspect the Entry set used for matching will be the same
>>>>>>> as
>>>>>>> the
>>>>>>> filter set, probably can provide a nice method signature to help that.
>>>>>>>
>>>>>>> Is there a way we can catch a class not found exception or similar and
>>>>>>> still
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> get the state of the superclass, or would this violate polymorphism,
>>>>>>>> method
>>>>>>>> overrides etc?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> Probably but again this is building clever code to try and compensate.
>>>>>>> It's
>>>>>>> the same as writing all the null handling if you will albeit hidden
>>>>>>> inside
>>>>>>> the River libraries.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> I guess the likelihood of having null values is reduced, although not
>>>>>>>> eliminated.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> Quite.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> If we compact the final array of Entry's so null entry's are removed,
>>>>>>>> the
>>>>>>>> client only needs to find the Entry's it's interested in, likely
>>>>>>>> ignoring
>>>>>>>> the others anyway.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> Sure but then the client has to explicitly construct filter code,
>>>>>>> albeit
>>>>>>> clean of null checks to go through the list of Entry's to find what
>>>>>>> it's
>>>>>>> interested in. Unless we return a HashMap(EntryClass.getName,
>>>>>>> EntryClass.instance) to make that easier. Even so, you're still
>>>>>>> returning
>>>>>>> a
>>>>>>> bunch of Entry's only some of which may be interesting to the
>>>>>>> client....
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> So the client needs to ensure the Entry's are resolvable from the
>>>>>>>> classpath.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> Absolutely - and the fact that it must means it can thus construct a
>>>>>>> proper
>>>>>>> filter list of Entrys of interest.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> I have thought about the case where a proxy uses the lookup service
>>>>>>>> and
>>>>>>>> has
>>>>>>>> Entry's in it's proxy classloader, but this could be subject to
>>>>>>>> security
>>>>>>>> issues, where two services with different Subject's end up sharing
>>>>>>>> state.
>>>>>>>>
>>>>>>>> We might also consider only including Entry's that don't have null
>>>>>>>> field
>>>>>>>> values, this puts the impetus on the implementer to consider the
>>>>>>>> Entry
>>>>>>>> field
>>>>>>>> classes.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> Again, I think we're venturing into attempting to imply intent by
>>>>>>> examining
>>>>>>> null fields. If the implementer makes a mistake in their analysis they
>>>>>>> end
>>>>>>> up with unexpected results come lookup. Personally I don't like this
>>>>>>> sort
>>>>>>> of
>>>>>>> thing, I like explicit methods to guide an implementer and eliminate
>>>>>>> opportunities for these kinds of confusion.
>>>>>>>
>>>>>>> That last bit is really why I'm down on trying to solve all these
>>>>>>> problems
>>>>>>> (including the bandwidth ones) with clever management of
>>>>>>> class-downloading,
>>>>>>> reflection and such. It's implicit, makes assumptions about usage that
>>>>>>> are
>>>>>>> difficult for users to account for. It also leads to complicated,
>>>>>>> brittle
>>>>>>> code with many an opportunity for dark-corners/unhandled edge cases
>>>>>>> that
>>>>>>> cause confusion and are difficult to identify and fix for River
>>>>>>> implementers
>>>>>>> and users.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Peter.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>
>>>>>>>               
>>>>>>             
>>>>>
>>>>>           
>>>>         
>>>
>>>       
>>     
>
>   

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

I was assuming a filter was inclusive by default. i.e. You specify just the
things you're interested in a client seeing.

There seems less value in removal given a client specifies its interests
somewhat in relation to what is on its classpath....

On 6 February 2011 11:45, Peter Firmstone <ji...@zeus.net.au> wrote:

> A filter could also collect, rather than just remove.
>
> Do you think query/filter builders should be specific to a particular Entry
> type?
>
>
> Cheers,
>
> Peter.
>
> Dan Creswell wrote:
>
>> Yep.
>>
>> On 5 February 2011 10:45, Peter Firmstone <ji...@zeus.net.au> wrote:
>>
>>
>>
>>> Sounds interesting, looks like a builder?
>>>
>>>
>>> Dan Creswell wrote:
>>>
>>>
>>>
>>>> I'm also figuring you might have some wise defaults that people could
>>>> add
>>>> to
>>>> for constructing a filter/matchset.
>>>>
>>>> Something like:
>>>>
>>>> DefaultEntrys.add(...).add(....)
>>>>
>>>> DefaultEntrys probably includes something like ServiceInfo, Name and
>>>> such
>>>> -
>>>> all stock options in the platform...
>>>>
>>>> On 5 February 2011 10:28, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>>>>>
>>>>> So we make the Entry's the client's interested in, the only Entry's
>>>>> that'll
>>>>> be unmarshalled, those and the ServiceID.
>>>>>
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Peter.
>>>>>
>>>>> Dan Creswell wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> ...
>>>>>>
>>>>>> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Dan Creswell wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> "ServiceClasspathSubItem is intended for client side filtering of
>>>>>>>> lookup
>>>>>>>> * service results prior to clients using a service, the lookup
>>>>>>>> service
>>>>>>>> * that implements this class, implements #getServiceItem(), so
>>>>>>>> clients
>>>>>>>> * can obtain a complete ServiceItem when required after filtering."
>>>>>>>>
>>>>>>>> I'm still wondering if it wouldn't be better for a client to send a
>>>>>>>> list
>>>>>>>> of
>>>>>>>> Entry's it's interested in seeing, separate from the set for
>>>>>>>> matching
>>>>>>>> to
>>>>>>>> the
>>>>>>>> Registrar. The Registrar can then return ServiceItems for the
>>>>>>>> matches
>>>>>>>> and
>>>>>>>> pre-filter the Entry's per item. Especially given:
>>>>>>>>
>>>>>>>> "Some fields in ServiceClasspathSubItem may be null, fields in
>>>>>>>> Entry's
>>>>>>>> may
>>>>>>>> * be null or even the service reference may be null, these fields
>>>>>>>> would
>>>>>>>> be
>>>>>>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>>>>>>> downloaded
>>>>>>>> * code or a remote codebase."
>>>>>>>>
>>>>>>>> I think, if we do the filtering as I suggest, much of the comment
>>>>>>>> re:
>>>>>>>> nulls
>>>>>>>> above still applies in implementation but ultimately, the client
>>>>>>>> will
>>>>>>>> pick
>>>>>>>> Entry's for which it has the classpath such that it always deals in
>>>>>>>> complete
>>>>>>>> Entry's and can have a level of predictability in respect of what is
>>>>>>>> or
>>>>>>>> is
>>>>>>>> not null. This might save clients having to do endless null checks
>>>>>>>> etc
>>>>>>>> that
>>>>>>>> can be a rich source of bugs and ugly code.
>>>>>>>>
>>>>>>>> We might need the filter list to include specific subclasses as
>>>>>>>> well....
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Hmm, null is a problem, we could still have fields missing subclass
>>>>>>> classes, which might prevent a subclass from unmarshalling.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Yeah, which is why I'm tempted to have the client be explicit about
>>>>>> which
>>>>>> Entry's (and that would include subclasses of those Entry's it can
>>>>>> tolerate). In this way the client can provide high-level explicit
>>>>>> guidance
>>>>>> as to what's right. No need for code to try and be clever and guess
>>>>>> the
>>>>>> client intent which is always ugly.
>>>>>>
>>>>>> Many times I suspect the Entry set used for matching will be the same
>>>>>> as
>>>>>> the
>>>>>> filter set, probably can provide a nice method signature to help that.
>>>>>>
>>>>>> Is there a way we can catch a class not found exception or similar and
>>>>>> still
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> get the state of the superclass, or would this violate polymorphism,
>>>>>>> method
>>>>>>> overrides etc?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Probably but again this is building clever code to try and compensate.
>>>>>> It's
>>>>>> the same as writing all the null handling if you will albeit hidden
>>>>>> inside
>>>>>> the River libraries.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I guess the likelihood of having null values is reduced, although not
>>>>>>> eliminated.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Quite.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> If we compact the final array of Entry's so null entry's are removed,
>>>>>>> the
>>>>>>> client only needs to find the Entry's it's interested in, likely
>>>>>>> ignoring
>>>>>>> the others anyway.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Sure but then the client has to explicitly construct filter code,
>>>>>> albeit
>>>>>> clean of null checks to go through the list of Entry's to find what
>>>>>> it's
>>>>>> interested in. Unless we return a HashMap(EntryClass.getName,
>>>>>> EntryClass.instance) to make that easier. Even so, you're still
>>>>>> returning
>>>>>> a
>>>>>> bunch of Entry's only some of which may be interesting to the
>>>>>> client....
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> So the client needs to ensure the Entry's are resolvable from the
>>>>>>> classpath.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Absolutely - and the fact that it must means it can thus construct a
>>>>>> proper
>>>>>> filter list of Entrys of interest.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> I have thought about the case where a proxy uses the lookup service
>>>>>>> and
>>>>>>> has
>>>>>>> Entry's in it's proxy classloader, but this could be subject to
>>>>>>> security
>>>>>>> issues, where two services with different Subject's end up sharing
>>>>>>> state.
>>>>>>>
>>>>>>> We might also consider only including Entry's that don't have null
>>>>>>> field
>>>>>>> values, this puts the impetus on the implementer to consider the
>>>>>>> Entry
>>>>>>> field
>>>>>>> classes.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Again, I think we're venturing into attempting to imply intent by
>>>>>> examining
>>>>>> null fields. If the implementer makes a mistake in their analysis they
>>>>>> end
>>>>>> up with unexpected results come lookup. Personally I don't like this
>>>>>> sort
>>>>>> of
>>>>>> thing, I like explicit methods to guide an implementer and eliminate
>>>>>> opportunities for these kinds of confusion.
>>>>>>
>>>>>> That last bit is really why I'm down on trying to solve all these
>>>>>> problems
>>>>>> (including the bandwidth ones) with clever management of
>>>>>> class-downloading,
>>>>>> reflection and such. It's implicit, makes assumptions about usage that
>>>>>> are
>>>>>> difficult for users to account for. It also leads to complicated,
>>>>>> brittle
>>>>>> code with many an opportunity for dark-corners/unhandled edge cases
>>>>>> that
>>>>>> cause confusion and are difficult to identify and fix for River
>>>>>> implementers
>>>>>> and users.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Peter.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

A filter could also collect, rather than just remove.

Do you think query/filter builders should be specific to a particular 
Entry type?

Cheers,

Peter.

Dan Creswell wrote:
> Yep.
>
> On 5 February 2011 10:45, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> Sounds interesting, looks like a builder?
>>
>>
>> Dan Creswell wrote:
>>
>>     
>>> I'm also figuring you might have some wise defaults that people could add
>>> to
>>> for constructing a filter/matchset.
>>>
>>> Something like:
>>>
>>> DefaultEntrys.add(...).add(....)
>>>
>>> DefaultEntrys probably includes something like ServiceInfo, Name and such
>>> -
>>> all stock options in the platform...
>>>
>>> On 5 February 2011 10:28, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>
>>>
>>>
>>>       
>>>> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>>>>
>>>> So we make the Entry's the client's interested in, the only Entry's
>>>> that'll
>>>> be unmarshalled, those and the ServiceID.
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Peter.
>>>>
>>>> Dan Creswell wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> ...
>>>>>
>>>>> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Dan Creswell wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> "ServiceClasspathSubItem is intended for client side filtering of
>>>>>>> lookup
>>>>>>> * service results prior to clients using a service, the lookup service
>>>>>>> * that implements this class, implements #getServiceItem(), so clients
>>>>>>> * can obtain a complete ServiceItem when required after filtering."
>>>>>>>
>>>>>>> I'm still wondering if it wouldn't be better for a client to send a
>>>>>>> list
>>>>>>> of
>>>>>>> Entry's it's interested in seeing, separate from the set for matching
>>>>>>> to
>>>>>>> the
>>>>>>> Registrar. The Registrar can then return ServiceItems for the matches
>>>>>>> and
>>>>>>> pre-filter the Entry's per item. Especially given:
>>>>>>>
>>>>>>> "Some fields in ServiceClasspathSubItem may be null, fields in Entry's
>>>>>>> may
>>>>>>> * be null or even the service reference may be null, these fields
>>>>>>> would
>>>>>>> be
>>>>>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>>>>>> downloaded
>>>>>>> * code or a remote codebase."
>>>>>>>
>>>>>>> I think, if we do the filtering as I suggest, much of the comment re:
>>>>>>> nulls
>>>>>>> above still applies in implementation but ultimately, the client will
>>>>>>> pick
>>>>>>> Entry's for which it has the classpath such that it always deals in
>>>>>>> complete
>>>>>>> Entry's and can have a level of predictability in respect of what is
>>>>>>> or
>>>>>>> is
>>>>>>> not null. This might save clients having to do endless null checks etc
>>>>>>> that
>>>>>>> can be a rich source of bugs and ugly code.
>>>>>>>
>>>>>>> We might need the filter list to include specific subclasses as
>>>>>>> well....
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Hmm, null is a problem, we could still have fields missing subclass
>>>>>> classes, which might prevent a subclass from unmarshalling.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> Yeah, which is why I'm tempted to have the client be explicit about
>>>>> which
>>>>> Entry's (and that would include subclasses of those Entry's it can
>>>>> tolerate). In this way the client can provide high-level explicit
>>>>> guidance
>>>>> as to what's right. No need for code to try and be clever and guess the
>>>>> client intent which is always ugly.
>>>>>
>>>>> Many times I suspect the Entry set used for matching will be the same as
>>>>> the
>>>>> filter set, probably can provide a nice method signature to help that.
>>>>>
>>>>> Is there a way we can catch a class not found exception or similar and
>>>>> still
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> get the state of the superclass, or would this violate polymorphism,
>>>>>> method
>>>>>> overrides etc?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> Probably but again this is building clever code to try and compensate.
>>>>> It's
>>>>> the same as writing all the null handling if you will albeit hidden
>>>>> inside
>>>>> the River libraries.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> I guess the likelihood of having null values is reduced, although not
>>>>>> eliminated.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> Quite.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> If we compact the final array of Entry's so null entry's are removed,
>>>>>> the
>>>>>> client only needs to find the Entry's it's interested in, likely
>>>>>> ignoring
>>>>>> the others anyway.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> Sure but then the client has to explicitly construct filter code, albeit
>>>>> clean of null checks to go through the list of Entry's to find what it's
>>>>> interested in. Unless we return a HashMap(EntryClass.getName,
>>>>> EntryClass.instance) to make that easier. Even so, you're still
>>>>> returning
>>>>> a
>>>>> bunch of Entry's only some of which may be interesting to the client....
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> So the client needs to ensure the Entry's are resolvable from the
>>>>>> classpath.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> Absolutely - and the fact that it must means it can thus construct a
>>>>> proper
>>>>> filter list of Entrys of interest.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> I have thought about the case where a proxy uses the lookup service and
>>>>>> has
>>>>>> Entry's in it's proxy classloader, but this could be subject to
>>>>>> security
>>>>>> issues, where two services with different Subject's end up sharing
>>>>>> state.
>>>>>>
>>>>>> We might also consider only including Entry's that don't have null
>>>>>> field
>>>>>> values, this puts the impetus on the implementer to consider the Entry
>>>>>> field
>>>>>> classes.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> Again, I think we're venturing into attempting to imply intent by
>>>>> examining
>>>>> null fields. If the implementer makes a mistake in their analysis they
>>>>> end
>>>>> up with unexpected results come lookup. Personally I don't like this
>>>>> sort
>>>>> of
>>>>> thing, I like explicit methods to guide an implementer and eliminate
>>>>> opportunities for these kinds of confusion.
>>>>>
>>>>> That last bit is really why I'm down on trying to solve all these
>>>>> problems
>>>>> (including the bandwidth ones) with clever management of
>>>>> class-downloading,
>>>>> reflection and such. It's implicit, makes assumptions about usage that
>>>>> are
>>>>> difficult for users to account for. It also leads to complicated,
>>>>> brittle
>>>>> code with many an opportunity for dark-corners/unhandled edge cases that
>>>>> cause confusion and are difficult to identify and fix for River
>>>>> implementers
>>>>> and users.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Cheers,
>>>>>>
>>>>>> Peter.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>
>>>>>           
>>>>         
>>>
>>>       
>>     
>
>   

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

Yep.

On 5 February 2011 10:45, Peter Firmstone <ji...@zeus.net.au> wrote:

> Sounds interesting, looks like a builder?
>
>
> Dan Creswell wrote:
>
>> I'm also figuring you might have some wise defaults that people could add
>> to
>> for constructing a filter/matchset.
>>
>> Something like:
>>
>> DefaultEntrys.add(...).add(....)
>>
>> DefaultEntrys probably includes something like ServiceInfo, Name and such
>> -
>> all stock options in the platform...
>>
>> On 5 February 2011 10:28, Peter Firmstone <ji...@zeus.net.au> wrote:
>>
>>
>>
>>> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>>>
>>> So we make the Entry's the client's interested in, the only Entry's
>>> that'll
>>> be unmarshalled, those and the ServiceID.
>>>
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>> Dan Creswell wrote:
>>>
>>>
>>>
>>>> ...
>>>>
>>>> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Dan Creswell wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> "ServiceClasspathSubItem is intended for client side filtering of
>>>>>> lookup
>>>>>> * service results prior to clients using a service, the lookup service
>>>>>> * that implements this class, implements #getServiceItem(), so clients
>>>>>> * can obtain a complete ServiceItem when required after filtering."
>>>>>>
>>>>>> I'm still wondering if it wouldn't be better for a client to send a
>>>>>> list
>>>>>> of
>>>>>> Entry's it's interested in seeing, separate from the set for matching
>>>>>> to
>>>>>> the
>>>>>> Registrar. The Registrar can then return ServiceItems for the matches
>>>>>> and
>>>>>> pre-filter the Entry's per item. Especially given:
>>>>>>
>>>>>> "Some fields in ServiceClasspathSubItem may be null, fields in Entry's
>>>>>> may
>>>>>> * be null or even the service reference may be null, these fields
>>>>>> would
>>>>>> be
>>>>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>>>>> downloaded
>>>>>> * code or a remote codebase."
>>>>>>
>>>>>> I think, if we do the filtering as I suggest, much of the comment re:
>>>>>> nulls
>>>>>> above still applies in implementation but ultimately, the client will
>>>>>> pick
>>>>>> Entry's for which it has the classpath such that it always deals in
>>>>>> complete
>>>>>> Entry's and can have a level of predictability in respect of what is
>>>>>> or
>>>>>> is
>>>>>> not null. This might save clients having to do endless null checks etc
>>>>>> that
>>>>>> can be a rich source of bugs and ugly code.
>>>>>>
>>>>>> We might need the filter list to include specific subclasses as
>>>>>> well....
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> Hmm, null is a problem, we could still have fields missing subclass
>>>>> classes, which might prevent a subclass from unmarshalling.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Yeah, which is why I'm tempted to have the client be explicit about
>>>> which
>>>> Entry's (and that would include subclasses of those Entry's it can
>>>> tolerate). In this way the client can provide high-level explicit
>>>> guidance
>>>> as to what's right. No need for code to try and be clever and guess the
>>>> client intent which is always ugly.
>>>>
>>>> Many times I suspect the Entry set used for matching will be the same as
>>>> the
>>>> filter set, probably can provide a nice method signature to help that.
>>>>
>>>> Is there a way we can catch a class not found exception or similar and
>>>> still
>>>>
>>>>
>>>>
>>>>
>>>>> get the state of the superclass, or would this violate polymorphism,
>>>>> method
>>>>> overrides etc?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Probably but again this is building clever code to try and compensate.
>>>> It's
>>>> the same as writing all the null handling if you will albeit hidden
>>>> inside
>>>> the River libraries.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> I guess the likelihood of having null values is reduced, although not
>>>>> eliminated.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Quite.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> If we compact the final array of Entry's so null entry's are removed,
>>>>> the
>>>>> client only needs to find the Entry's it's interested in, likely
>>>>> ignoring
>>>>> the others anyway.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Sure but then the client has to explicitly construct filter code, albeit
>>>> clean of null checks to go through the list of Entry's to find what it's
>>>> interested in. Unless we return a HashMap(EntryClass.getName,
>>>> EntryClass.instance) to make that easier. Even so, you're still
>>>> returning
>>>> a
>>>> bunch of Entry's only some of which may be interesting to the client....
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> So the client needs to ensure the Entry's are resolvable from the
>>>>> classpath.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Absolutely - and the fact that it must means it can thus construct a
>>>> proper
>>>> filter list of Entrys of interest.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> I have thought about the case where a proxy uses the lookup service and
>>>>> has
>>>>> Entry's in it's proxy classloader, but this could be subject to
>>>>> security
>>>>> issues, where two services with different Subject's end up sharing
>>>>> state.
>>>>>
>>>>> We might also consider only including Entry's that don't have null
>>>>> field
>>>>> values, this puts the impetus on the implementer to consider the Entry
>>>>> field
>>>>> classes.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> Again, I think we're venturing into attempting to imply intent by
>>>> examining
>>>> null fields. If the implementer makes a mistake in their analysis they
>>>> end
>>>> up with unexpected results come lookup. Personally I don't like this
>>>> sort
>>>> of
>>>> thing, I like explicit methods to guide an implementer and eliminate
>>>> opportunities for these kinds of confusion.
>>>>
>>>> That last bit is really why I'm down on trying to solve all these
>>>> problems
>>>> (including the bandwidth ones) with clever management of
>>>> class-downloading,
>>>> reflection and such. It's implicit, makes assumptions about usage that
>>>> are
>>>> difficult for users to account for. It also leads to complicated,
>>>> brittle
>>>> code with many an opportunity for dark-corners/unhandled edge cases that
>>>> cause confusion and are difficult to identify and fix for River
>>>> implementers
>>>> and users.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Cheers,
>>>>>
>>>>> Peter.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Sounds interesting, looks like a builder?

Dan Creswell wrote:
> I'm also figuring you might have some wise defaults that people could add to
> for constructing a filter/matchset.
>
> Something like:
>
> DefaultEntrys.add(...).add(....)
>
> DefaultEntrys probably includes something like ServiceInfo, Name and such -
> all stock options in the platform...
>
> On 5 February 2011 10:28, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>>
>> So we make the Entry's the client's interested in, the only Entry's that'll
>> be unmarshalled, those and the ServiceID.
>>
>>
>> Cheers,
>>
>> Peter.
>>
>> Dan Creswell wrote:
>>
>>     
>>> ...
>>>
>>> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>
>>>
>>>
>>>       
>>>> Dan Creswell wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> "ServiceClasspathSubItem is intended for client side filtering of lookup
>>>>> * service results prior to clients using a service, the lookup service
>>>>> * that implements this class, implements #getServiceItem(), so clients
>>>>> * can obtain a complete ServiceItem when required after filtering."
>>>>>
>>>>> I'm still wondering if it wouldn't be better for a client to send a list
>>>>> of
>>>>> Entry's it's interested in seeing, separate from the set for matching to
>>>>> the
>>>>> Registrar. The Registrar can then return ServiceItems for the matches
>>>>> and
>>>>> pre-filter the Entry's per item. Especially given:
>>>>>
>>>>> "Some fields in ServiceClasspathSubItem may be null, fields in Entry's
>>>>> may
>>>>> * be null or even the service reference may be null, these fields would
>>>>> be
>>>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>>>> downloaded
>>>>> * code or a remote codebase."
>>>>>
>>>>> I think, if we do the filtering as I suggest, much of the comment re:
>>>>> nulls
>>>>> above still applies in implementation but ultimately, the client will
>>>>> pick
>>>>> Entry's for which it has the classpath such that it always deals in
>>>>> complete
>>>>> Entry's and can have a level of predictability in respect of what is or
>>>>> is
>>>>> not null. This might save clients having to do endless null checks etc
>>>>> that
>>>>> can be a rich source of bugs and ugly code.
>>>>>
>>>>> We might need the filter list to include specific subclasses as well....
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> Hmm, null is a problem, we could still have fields missing subclass
>>>> classes, which might prevent a subclass from unmarshalling.
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> Yeah, which is why I'm tempted to have the client be explicit about which
>>> Entry's (and that would include subclasses of those Entry's it can
>>> tolerate). In this way the client can provide high-level explicit guidance
>>> as to what's right. No need for code to try and be clever and guess the
>>> client intent which is always ugly.
>>>
>>> Many times I suspect the Entry set used for matching will be the same as
>>> the
>>> filter set, probably can provide a nice method signature to help that.
>>>
>>> Is there a way we can catch a class not found exception or similar and
>>> still
>>>
>>>
>>>       
>>>> get the state of the superclass, or would this violate polymorphism,
>>>> method
>>>> overrides etc?
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> Probably but again this is building clever code to try and compensate.
>>> It's
>>> the same as writing all the null handling if you will albeit hidden inside
>>> the River libraries.
>>>
>>>
>>>
>>>
>>>       
>>>> I guess the likelihood of having null values is reduced, although not
>>>> eliminated.
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> Quite.
>>>
>>>
>>>
>>>
>>>       
>>>> If we compact the final array of Entry's so null entry's are removed, the
>>>> client only needs to find the Entry's it's interested in, likely ignoring
>>>> the others anyway.
>>>>
>>>>
>>>>
>>>>         
>>> Sure but then the client has to explicitly construct filter code, albeit
>>> clean of null checks to go through the list of Entry's to find what it's
>>> interested in. Unless we return a HashMap(EntryClass.getName,
>>> EntryClass.instance) to make that easier. Even so, you're still returning
>>> a
>>> bunch of Entry's only some of which may be interesting to the client....
>>>
>>>
>>>
>>>
>>>       
>>>> So the client needs to ensure the Entry's are resolvable from the
>>>> classpath.
>>>>
>>>>
>>>>
>>>>
>>>>         
>>> Absolutely - and the fact that it must means it can thus construct a
>>> proper
>>> filter list of Entrys of interest.
>>>
>>>
>>>
>>>
>>>       
>>>> I have thought about the case where a proxy uses the lookup service and
>>>> has
>>>> Entry's in it's proxy classloader, but this could be subject to security
>>>> issues, where two services with different Subject's end up sharing state.
>>>>
>>>> We might also consider only including Entry's that don't have null field
>>>> values, this puts the impetus on the implementer to consider the Entry
>>>> field
>>>> classes.
>>>>
>>>>
>>>>
>>>>         
>>> Again, I think we're venturing into attempting to imply intent by
>>> examining
>>> null fields. If the implementer makes a mistake in their analysis they end
>>> up with unexpected results come lookup. Personally I don't like this sort
>>> of
>>> thing, I like explicit methods to guide an implementer and eliminate
>>> opportunities for these kinds of confusion.
>>>
>>> That last bit is really why I'm down on trying to solve all these problems
>>> (including the bandwidth ones) with clever management of
>>> class-downloading,
>>> reflection and such. It's implicit, makes assumptions about usage that are
>>> difficult for users to account for. It also leads to complicated, brittle
>>> code with many an opportunity for dark-corners/unhandled edge cases that
>>> cause confusion and are difficult to identify and fix for River
>>> implementers
>>> and users.
>>>
>>>
>>>
>>>
>>>       
>>>> Cheers,
>>>>
>>>> Peter.
>>>>
>>>>
>>>>
>>>>         
>>>
>>>       
>>     
>
>   

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

I'm also figuring you might have some wise defaults that people could add to
for constructing a filter/matchset.

Something like:

DefaultEntrys.add(...).add(....)

DefaultEntrys probably includes something like ServiceInfo, Name and such -
all stock options in the platform...

On 5 February 2011 10:28, Peter Firmstone <ji...@zeus.net.au> wrote:

> Ok, good reasoning, we can keep the Entry's explicit, in the call.
>
> So we make the Entry's the client's interested in, the only Entry's that'll
> be unmarshalled, those and the ServiceID.
>
>
> Cheers,
>
> Peter.
>
> Dan Creswell wrote:
>
>> ...
>>
>> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>>
>>
>>
>>> Dan Creswell wrote:
>>>
>>>
>>>
>>>> "ServiceClasspathSubItem is intended for client side filtering of lookup
>>>> * service results prior to clients using a service, the lookup service
>>>> * that implements this class, implements #getServiceItem(), so clients
>>>> * can obtain a complete ServiceItem when required after filtering."
>>>>
>>>> I'm still wondering if it wouldn't be better for a client to send a list
>>>> of
>>>> Entry's it's interested in seeing, separate from the set for matching to
>>>> the
>>>> Registrar. The Registrar can then return ServiceItems for the matches
>>>> and
>>>> pre-filter the Entry's per item. Especially given:
>>>>
>>>> "Some fields in ServiceClasspathSubItem may be null, fields in Entry's
>>>> may
>>>> * be null or even the service reference may be null, these fields would
>>>> be
>>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>>> downloaded
>>>> * code or a remote codebase."
>>>>
>>>> I think, if we do the filtering as I suggest, much of the comment re:
>>>> nulls
>>>> above still applies in implementation but ultimately, the client will
>>>> pick
>>>> Entry's for which it has the classpath such that it always deals in
>>>> complete
>>>> Entry's and can have a level of predictability in respect of what is or
>>>> is
>>>> not null. This might save clients having to do endless null checks etc
>>>> that
>>>> can be a rich source of bugs and ugly code.
>>>>
>>>> We might need the filter list to include specific subclasses as well....
>>>>
>>>>
>>>>
>>>>
>>>>
>>> Hmm, null is a problem, we could still have fields missing subclass
>>> classes, which might prevent a subclass from unmarshalling.
>>>
>>>
>>>
>>>
>> Yeah, which is why I'm tempted to have the client be explicit about which
>> Entry's (and that would include subclasses of those Entry's it can
>> tolerate). In this way the client can provide high-level explicit guidance
>> as to what's right. No need for code to try and be clever and guess the
>> client intent which is always ugly.
>>
>> Many times I suspect the Entry set used for matching will be the same as
>> the
>> filter set, probably can provide a nice method signature to help that.
>>
>> Is there a way we can catch a class not found exception or similar and
>> still
>>
>>
>>> get the state of the superclass, or would this violate polymorphism,
>>> method
>>> overrides etc?
>>>
>>>
>>>
>>>
>> Probably but again this is building clever code to try and compensate.
>> It's
>> the same as writing all the null handling if you will albeit hidden inside
>> the River libraries.
>>
>>
>>
>>
>>> I guess the likelihood of having null values is reduced, although not
>>> eliminated.
>>>
>>>
>>>
>>>
>> Quite.
>>
>>
>>
>>
>>> If we compact the final array of Entry's so null entry's are removed, the
>>> client only needs to find the Entry's it's interested in, likely ignoring
>>> the others anyway.
>>>
>>>
>>>
>>
>> Sure but then the client has to explicitly construct filter code, albeit
>> clean of null checks to go through the list of Entry's to find what it's
>> interested in. Unless we return a HashMap(EntryClass.getName,
>> EntryClass.instance) to make that easier. Even so, you're still returning
>> a
>> bunch of Entry's only some of which may be interesting to the client....
>>
>>
>>
>>
>>> So the client needs to ensure the Entry's are resolvable from the
>>> classpath.
>>>
>>>
>>>
>>>
>> Absolutely - and the fact that it must means it can thus construct a
>> proper
>> filter list of Entrys of interest.
>>
>>
>>
>>
>>> I have thought about the case where a proxy uses the lookup service and
>>> has
>>> Entry's in it's proxy classloader, but this could be subject to security
>>> issues, where two services with different Subject's end up sharing state.
>>>
>>> We might also consider only including Entry's that don't have null field
>>> values, this puts the impetus on the implementer to consider the Entry
>>> field
>>> classes.
>>>
>>>
>>>
>>
>> Again, I think we're venturing into attempting to imply intent by
>> examining
>> null fields. If the implementer makes a mistake in their analysis they end
>> up with unexpected results come lookup. Personally I don't like this sort
>> of
>> thing, I like explicit methods to guide an implementer and eliminate
>> opportunities for these kinds of confusion.
>>
>> That last bit is really why I'm down on trying to solve all these problems
>> (including the bandwidth ones) with clever management of
>> class-downloading,
>> reflection and such. It's implicit, makes assumptions about usage that are
>> difficult for users to account for. It also leads to complicated, brittle
>> code with many an opportunity for dark-corners/unhandled edge cases that
>> cause confusion and are difficult to identify and fix for River
>> implementers
>> and users.
>>
>>
>>
>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>>
>>>
>>
>>
>>
>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Ok, good reasoning, we can keep the Entry's explicit, in the call.

So we make the Entry's the client's interested in, the only Entry's 
that'll be unmarshalled, those and the ServiceID.

Cheers,

Peter.

Dan Creswell wrote:
> ...
>
> On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> Dan Creswell wrote:
>>
>>     
>>> "ServiceClasspathSubItem is intended for client side filtering of lookup
>>> * service results prior to clients using a service, the lookup service
>>> * that implements this class, implements #getServiceItem(), so clients
>>> * can obtain a complete ServiceItem when required after filtering."
>>>
>>> I'm still wondering if it wouldn't be better for a client to send a list
>>> of
>>> Entry's it's interested in seeing, separate from the set for matching to
>>> the
>>> Registrar. The Registrar can then return ServiceItems for the matches and
>>> pre-filter the Entry's per item. Especially given:
>>>
>>> "Some fields in ServiceClasspathSubItem may be null, fields in Entry's may
>>> * be null or even the service reference may be null, these fields would be
>>> * non-null in a ServiceItem that resolves classes from dynamicly
>>> downloaded
>>> * code or a remote codebase."
>>>
>>> I think, if we do the filtering as I suggest, much of the comment re:
>>> nulls
>>> above still applies in implementation but ultimately, the client will pick
>>> Entry's for which it has the classpath such that it always deals in
>>> complete
>>> Entry's and can have a level of predictability in respect of what is or is
>>> not null. This might save clients having to do endless null checks etc
>>> that
>>> can be a rich source of bugs and ugly code.
>>>
>>> We might need the filter list to include specific subclasses as well....
>>>
>>>
>>>
>>>       
>> Hmm, null is a problem, we could still have fields missing subclass
>> classes, which might prevent a subclass from unmarshalling.
>>
>>
>>     
> Yeah, which is why I'm tempted to have the client be explicit about which
> Entry's (and that would include subclasses of those Entry's it can
> tolerate). In this way the client can provide high-level explicit guidance
> as to what's right. No need for code to try and be clever and guess the
> client intent which is always ugly.
>
> Many times I suspect the Entry set used for matching will be the same as the
> filter set, probably can provide a nice method signature to help that.
>
> Is there a way we can catch a class not found exception or similar and still
>   
>> get the state of the superclass, or would this violate polymorphism, method
>> overrides etc?
>>
>>
>>     
> Probably but again this is building clever code to try and compensate. It's
> the same as writing all the null handling if you will albeit hidden inside
> the River libraries.
>
>
>   
>> I guess the likelihood of having null values is reduced, although not
>> eliminated.
>>
>>
>>     
> Quite.
>
>
>   
>> If we compact the final array of Entry's so null entry's are removed, the
>> client only needs to find the Entry's it's interested in, likely ignoring
>> the others anyway.
>>
>>     
>
> Sure but then the client has to explicitly construct filter code, albeit
> clean of null checks to go through the list of Entry's to find what it's
> interested in. Unless we return a HashMap(EntryClass.getName,
> EntryClass.instance) to make that easier. Even so, you're still returning a
> bunch of Entry's only some of which may be interesting to the client....
>
>
>   
>> So the client needs to ensure the Entry's are resolvable from the
>> classpath.
>>
>>
>>     
> Absolutely - and the fact that it must means it can thus construct a proper
> filter list of Entrys of interest.
>
>
>   
>> I have thought about the case where a proxy uses the lookup service and has
>> Entry's in it's proxy classloader, but this could be subject to security
>> issues, where two services with different Subject's end up sharing state.
>>
>> We might also consider only including Entry's that don't have null field
>> values, this puts the impetus on the implementer to consider the Entry field
>> classes.
>>
>>     
>
> Again, I think we're venturing into attempting to imply intent by examining
> null fields. If the implementer makes a mistake in their analysis they end
> up with unexpected results come lookup. Personally I don't like this sort of
> thing, I like explicit methods to guide an implementer and eliminate
> opportunities for these kinds of confusion.
>
> That last bit is really why I'm down on trying to solve all these problems
> (including the bandwidth ones) with clever management of class-downloading,
> reflection and such. It's implicit, makes assumptions about usage that are
> difficult for users to account for. It also leads to complicated, brittle
> code with many an opportunity for dark-corners/unhandled edge cases that
> cause confusion and are difficult to identify and fix for River implementers
> and users.
>
>
>   
>> Cheers,
>>
>> Peter.
>>
>>     
>
>   

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

How's this?

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.river.api.lookup;

import java.io.IOException;
import net.jini.core.lookup.ServiceRegistrar;
import net.jini.core.lookup.ServiceTemplate;
import net.jini.core.lookup.ServiceItem;
import org.apache.river.api.util.ResultStream;

/**
 * Defines an extension interface to the lookup service, for use on 
large or
 * global networks such as the internet or low bandwidth networks. 
 * The interface is not a remote interface; each implementation of the
 * lookup service exports proxy objects that implement the
 * StreamServiceRegistrar interface local to the client, using an
 * implementation-specific protocol to communicate with the actual remote
 * server.  All of the proxy methods obey normal RMI remote interface
 * semantics except where explicitly noted.  Two proxy objects are equal if
 * they are proxies for the same lookup service.  Every method invocation
 * (on both StreamServiceRegistrar and ServiceRegistration) is atomic with
 * respect to other invocations.
 *
 * The StreamServiceRegistrar is intended to perform the same function
 * as the ServiceRegistrar, but with the ability to return results as a
 * stream, so memory consumption is minimised at the client and network
 * communication is minimised between the client and lookup service server.
 *
 * All clients utilising ServiceRegistrar, should switch to the
 * StreamServiceRegistrar.
 *
 * @see ServiceRegistrar
 * @see PortableServiceRegistrar
 * @see ServiceRegistration
 * @author Peter Firmstone
 * @since 2.2.0
 */
public interface StreamServiceRegistrar extends ServiceRegistrar{

    /**
     * Returns a ResultStream that provides access to 
ServiceClasspathSubItem
     * instances.  The ResultStream terminates with a null value.  The 
result
     * stream may be infinite, or limited by an integer limit value.
     *
     * A ServiceClasspathSubItem implementation instance is a 
ServiceItem that
     * contains only Objects that are resolvable on the local classpath,
     * this is useful for clients to perform filtering before requiring a
     * download of the actual ServiceItem.
     *
     * The ResultStream should be closed once the desired service has been
     * found, or services have been processed.
     *
     * @param tmpl template to match
     * specified template
     *
     * @param maxBatchSize held locally, larger batch sizes reduce network
     * traffic, but may delay processing locally depending on 
implementation.
     * @param limit - Zero for infinite, otherwise limits the number of 
matching
     * results.
     * @return ResultStream containing ServiceItem's
     * @throws java.io.IOException
     * @see ServiceItem
     * @see ServiceClasspathSubItem
     * @see ResultStream
     * @see ServiceResultStreamFilter
     * @see ResultStreamUnmarshaller
     * @since 2.2.0
     */
    ResultStream lookup(ServiceTemplate tmpl, Class[] entryClasses,
            int maxBatchSize, int limit)  throws IOException;
}

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

...

On 5 February 2011 09:48, Peter Firmstone <ji...@zeus.net.au> wrote:

> Dan Creswell wrote:
>
>> "ServiceClasspathSubItem is intended for client side filtering of lookup
>> * service results prior to clients using a service, the lookup service
>> * that implements this class, implements #getServiceItem(), so clients
>> * can obtain a complete ServiceItem when required after filtering."
>>
>> I'm still wondering if it wouldn't be better for a client to send a list
>> of
>> Entry's it's interested in seeing, separate from the set for matching to
>> the
>> Registrar. The Registrar can then return ServiceItems for the matches and
>> pre-filter the Entry's per item. Especially given:
>>
>> "Some fields in ServiceClasspathSubItem may be null, fields in Entry's may
>> * be null or even the service reference may be null, these fields would be
>> * non-null in a ServiceItem that resolves classes from dynamicly
>> downloaded
>> * code or a remote codebase."
>>
>> I think, if we do the filtering as I suggest, much of the comment re:
>> nulls
>> above still applies in implementation but ultimately, the client will pick
>> Entry's for which it has the classpath such that it always deals in
>> complete
>> Entry's and can have a level of predictability in respect of what is or is
>> not null. This might save clients having to do endless null checks etc
>> that
>> can be a rich source of bugs and ugly code.
>>
>> We might need the filter list to include specific subclasses as well....
>>
>>
>>
> Hmm, null is a problem, we could still have fields missing subclass
> classes, which might prevent a subclass from unmarshalling.
>
>
Yeah, which is why I'm tempted to have the client be explicit about which
Entry's (and that would include subclasses of those Entry's it can
tolerate). In this way the client can provide high-level explicit guidance
as to what's right. No need for code to try and be clever and guess the
client intent which is always ugly.

Many times I suspect the Entry set used for matching will be the same as the
filter set, probably can provide a nice method signature to help that.

Is there a way we can catch a class not found exception or similar and still
> get the state of the superclass, or would this violate polymorphism, method
> overrides etc?
>
>
Probably but again this is building clever code to try and compensate. It's
the same as writing all the null handling if you will albeit hidden inside
the River libraries.


> I guess the likelihood of having null values is reduced, although not
> eliminated.
>
>
Quite.


> If we compact the final array of Entry's so null entry's are removed, the
> client only needs to find the Entry's it's interested in, likely ignoring
> the others anyway.
>

Sure but then the client has to explicitly construct filter code, albeit
clean of null checks to go through the list of Entry's to find what it's
interested in. Unless we return a HashMap(EntryClass.getName,
EntryClass.instance) to make that easier. Even so, you're still returning a
bunch of Entry's only some of which may be interesting to the client....


>
> So the client needs to ensure the Entry's are resolvable from the
> classpath.
>
>
Absolutely - and the fact that it must means it can thus construct a proper
filter list of Entrys of interest.


> I have thought about the case where a proxy uses the lookup service and has
> Entry's in it's proxy classloader, but this could be subject to security
> issues, where two services with different Subject's end up sharing state.
>
> We might also consider only including Entry's that don't have null field
> values, this puts the impetus on the implementer to consider the Entry field
> classes.
>

Again, I think we're venturing into attempting to imply intent by examining
null fields. If the implementer makes a mistake in their analysis they end
up with unexpected results come lookup. Personally I don't like this sort of
thing, I like explicit methods to guide an implementer and eliminate
opportunities for these kinds of confusion.

That last bit is really why I'm down on trying to solve all these problems
(including the bandwidth ones) with clever management of class-downloading,
reflection and such. It's implicit, makes assumptions about usage that are
difficult for users to account for. It also leads to complicated, brittle
code with many an opportunity for dark-corners/unhandled edge cases that
cause confusion and are difficult to identify and fix for River implementers
and users.


>
> Cheers,
>
> Peter.
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Dan Creswell wrote:
> "ServiceClasspathSubItem is intended for client side filtering of lookup
> * service results prior to clients using a service, the lookup service
> * that implements this class, implements #getServiceItem(), so clients
> * can obtain a complete ServiceItem when required after filtering."
>
> I'm still wondering if it wouldn't be better for a client to send a list of
> Entry's it's interested in seeing, separate from the set for matching to the
> Registrar. The Registrar can then return ServiceItems for the matches and
> pre-filter the Entry's per item. Especially given:
>
> "Some fields in ServiceClasspathSubItem may be null, fields in Entry's may
> * be null or even the service reference may be null, these fields would be
> * non-null in a ServiceItem that resolves classes from dynamicly downloaded
> * code or a remote codebase."
>
> I think, if we do the filtering as I suggest, much of the comment re: nulls
> above still applies in implementation but ultimately, the client will pick
> Entry's for which it has the classpath such that it always deals in complete
> Entry's and can have a level of predictability in respect of what is or is
> not null. This might save clients having to do endless null checks etc that
> can be a rich source of bugs and ugly code.
>
> We might need the filter list to include specific subclasses as well....
>
>   
Hmm, null is a problem, we could still have fields missing subclass 
classes, which might prevent a subclass from unmarshalling.

Is there a way we can catch a class not found exception or similar and 
still get the state of the superclass, or would this violate 
polymorphism, method overrides etc?

I guess the likelihood of having null values is reduced, although not 
eliminated.

If we compact the final array of Entry's so null entry's are removed, 
the client only needs to find the Entry's it's interested in, likely 
ignoring the others anyway.

So the client needs to ensure the Entry's are resolvable from the classpath.

I have thought about the case where a proxy uses the lookup service and 
has Entry's in it's proxy classloader, but this could be subject to 
security issues, where two services with different Subject's end up 
sharing state.

We might also consider only including Entry's that don't have null field 
values, this puts the impetus on the implementer to consider the Entry 
field classes.

Cheers,

Peter.

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

"ServiceClasspathSubItem is intended for client side filtering of lookup
* service results prior to clients using a service, the lookup service
* that implements this class, implements #getServiceItem(), so clients
* can obtain a complete ServiceItem when required after filtering."

I'm still wondering if it wouldn't be better for a client to send a list of
Entry's it's interested in seeing, separate from the set for matching to the
Registrar. The Registrar can then return ServiceItems for the matches and
pre-filter the Entry's per item. Especially given:

"Some fields in ServiceClasspathSubItem may be null, fields in Entry's may
* be null or even the service reference may be null, these fields would be
* non-null in a ServiceItem that resolves classes from dynamicly downloaded
* code or a remote codebase."

I think, if we do the filtering as I suggest, much of the comment re: nulls
above still applies in implementation but ultimately, the client will pick
Entry's for which it has the classpath such that it always deals in complete
Entry's and can have a level of predictability in respect of what is or is
not null. This might save clients having to do endless null checks etc that
can be a rich source of bugs and ugly code.

We might need the filter list to include specific subclasses as well....

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

By only using the classpath to resolve classes, the caller no longer 
needs to provide Entry classes to StreamServiceRegistrar either.

Revised class below, the javadoc, class and method name have been changed.

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.river.api.lookup;

import java.net.URI;
import java.security.CodeSource;
import net.jini.core.entry.Entry;
import net.jini.core.lookup.ServiceID;
import net.jini.core.lookup.ServiceItem;

/**
 * ServiceClasspathSubItem is intended for client side filtering of lookup
 * service results prior to clients using a service, the lookup service
 * that implements this class, implements #getServiceItem(), so clients
 * can obtain a complete ServiceItem when required after filtering.
 *
 * ServiceClasspathSubItem extends ServiceItem and can be used anywhere a
 * ServiceItem is required for querying or inspecting Entry fields that are
 * resolvable from the local classpath.  If dynamically downloaded code is
 * required, Remote or Serializable object references are not resolved,
 * instead, such fields are set to null to avoid codebase download.
 *
 * ServiceClasspathSubItem inherits all fields from ServiceItem.
 *
 * Some fields in ServiceClasspathSubItem may be null, fields in Entry's 
may
 * be null or even the service reference may be null, these fields would be
 * non-null in a ServiceItem that resolves classes from dynamicly 
downloaded
 * code or a remote codebase.
 *
 * The serviceID field shall be non-null always.
 *
 * ServiceItem's toString() method will return a different result for
 * ServiceClasspathSubItem instances.
 *
 * When required, a new ServiceItem that is unmarshalled
 * using remote codebases and dynamicly downloaded code can be obtained
 * by calling #getServiceItem().
 *
 * @author Peter Firmstone.
 */
public abstract class ServiceClasspathSubItem extends ServiceItem{
    private static final long SerialVersionUID = 1L;
    protected ServiceClasspathSubItem(ServiceID id, Entry[] 
unmarshalledEntries){
        super(id, (Object) null, unmarshalledEntries);
    }
   
    /* Default constructor for serializable sub class.
     */
    protected ServiceClasspathSubItem(){
        super(null, null, null);
    }
    /**
     * Using remote and local code as required getServiceItem returns a
     * new ServiceItem.
     *
     * The returned ServiceItem must not be an instance of this class.
     *
     * @return ServiceItem, totally unmarshalled, using remote codebase 
resources
     * in addition to any local classpath or resources.
     */
    public abstract ServiceItem getServiceItem();
}

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

It is a container object, it extends ServiceItem, it's essentially has 
identical state (or a subset of it), but unlike a ServiceItem, no 
attempt has been made to use or download remote codebases, only the 
classpath is used to resolve and unmarshall any objects contained in the 
ServiceItem.  Object references are set to null if their classes cannot 
be resolved locally, including the proxy, any Entry's or Entry fields.

A single method is provided to resolve the ServiceItem using remote 
codebases in addition to the classpath, this method returns a new 
ServiceItem.

The intent of the class is to provide access to Entry's for complex 
queries, (<>|&) performed locally, without downloading any codebases.  
Once the client has found a suitable service it can be retrieved after a 
codebase download and unmarshalling.

The implementer may also decide to avoid downloading the marshalled 
state of the service proxy, in addition to its codebase.

Other ideas:

PseudoServiceItem

PartialServiceItem

ServiceItemLocalSubset

Cheers,

Peter.

MICHAEL MCGRADY wrote:
> Could you state in a sentence what the class does?  Then we can all suggest possible names?
>
> MG
>
>
> On Feb 3, 2011, at 1:40 PM, Peter Firmstone wrote:
>
>   
>> Or one of these?
>>
>> ClasspathFencedServiceItem
>>
>> ClaspathBoundServiceItem
>>
>> ClasspathConstrainedServiceItem
>>
>> Dan Creswell wrote:
>>     
>>> Thing is the ServiceItem is not the thing being resolved....
>>>
>>> On 3 February 2011 09:59, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>
>>>  
>>>       
>>>> How about LocallyResolvedServiceItem?
>>>>
>>>>
>>>> Dan Creswell wrote:
>>>>
>>>>    
>>>>         
>>>>> Sure, the method does indeed unmarshall but within specific constraints
>>>>> and
>>>>> those constraints are defined by the enclosing class (an instance of
>>>>> ServiceItem in this case).
>>>>>
>>>>> So I'm thinking we're really talking about is a ServiceItem that pulls
>>>>> from
>>>>> classpath so maybe LocallyResolvingServiceItem (not entirely keen on that
>>>>> but explains what's happening).
>>>>>
>>>>> On some level there's a bit of me that screams we have a broken design if
>>>>> we
>>>>> can't get decent names.....
>>>>>
>>>>> On 3 February 2011 09:10, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>>>
>>>>>
>>>>>
>>>>>      
>>>>>           
>>>>>> Dan Creswell wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>        
>>>>>>             
>>>>>>> "In summary MarshalledServiceItem is a class that only resolves local
>>>>>>> code.
>>>>>>> Interestingly MarshalledServiceItem, might not be an appropriate name,
>>>>>>> perhaps LocalServiceItem to indicate that only local code is utilised,
>>>>>>> on
>>>>>>> this reasoning, I can't see any objection to reflective proxy's being
>>>>>>> non
>>>>>>> null either."
>>>>>>>
>>>>>>> Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
>>>>>>> what's going on.....
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>          
>>>>>>>               
>>>>>> The method unmarshall, sort of makes sense, can you think of a better
>>>>>> name?
>>>>>>
>>>>>>
>>>>>>
>>>>>>        
>>>>>>             
>>>>>      
>>>>>           
>>>>    
>>>>         
>>>  
>>>       
>
> Michael McGrady
> Chief Architect
> Topia Technology, Inc.
> Cel 1.253.720.3365
> Work 1.253.572.9712 extension 2037
> mmcgrady@topiatechnology.com
>
>
>
>
>   

Re: MarshalledServiceItem

[MICHAEL MCGRADY <mm...@topiatechnology.com>]

Could you state in a sentence what the class does?  Then we can all suggest possible names?

MG


On Feb 3, 2011, at 1:40 PM, Peter Firmstone wrote:

> Or one of these?
> 
> ClasspathFencedServiceItem
> 
> ClaspathBoundServiceItem
> 
> ClasspathConstrainedServiceItem
> 
> Dan Creswell wrote:
>> Thing is the ServiceItem is not the thing being resolved....
>> 
>> On 3 February 2011 09:59, Peter Firmstone <ji...@zeus.net.au> wrote:
>> 
>>  
>>> How about LocallyResolvedServiceItem?
>>> 
>>> 
>>> Dan Creswell wrote:
>>> 
>>>    
>>>> Sure, the method does indeed unmarshall but within specific constraints
>>>> and
>>>> those constraints are defined by the enclosing class (an instance of
>>>> ServiceItem in this case).
>>>> 
>>>> So I'm thinking we're really talking about is a ServiceItem that pulls
>>>> from
>>>> classpath so maybe LocallyResolvingServiceItem (not entirely keen on that
>>>> but explains what's happening).
>>>> 
>>>> On some level there's a bit of me that screams we have a broken design if
>>>> we
>>>> can't get decent names.....
>>>> 
>>>> On 3 February 2011 09:10, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>> 
>>>> 
>>>> 
>>>>      
>>>>> Dan Creswell wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>>        
>>>>>> "In summary MarshalledServiceItem is a class that only resolves local
>>>>>> code.
>>>>>> Interestingly MarshalledServiceItem, might not be an appropriate name,
>>>>>> perhaps LocalServiceItem to indicate that only local code is utilised,
>>>>>> on
>>>>>> this reasoning, I can't see any objection to reflective proxy's being
>>>>>> non
>>>>>> null either."
>>>>>> 
>>>>>> Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
>>>>>> what's going on.....
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>          
>>>>> The method unmarshall, sort of makes sense, can you think of a better
>>>>> name?
>>>>> 
>>>>> 
>>>>> 
>>>>>        
>>>> 
>>>>      
>>>    
>> 
>>  
> 

Michael McGrady
Chief Architect
Topia Technology, Inc.
Cel 1.253.720.3365
Work 1.253.572.9712 extension 2037
mmcgrady@topiatechnology.com

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Or one of these?

ClasspathFencedServiceItem

ClaspathBoundServiceItem

ClasspathConstrainedServiceItem

Dan Creswell wrote:
> Thing is the ServiceItem is not the thing being resolved....
>
> On 3 February 2011 09:59, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> How about LocallyResolvedServiceItem?
>>
>>
>> Dan Creswell wrote:
>>
>>     
>>> Sure, the method does indeed unmarshall but within specific constraints
>>> and
>>> those constraints are defined by the enclosing class (an instance of
>>> ServiceItem in this case).
>>>
>>> So I'm thinking we're really talking about is a ServiceItem that pulls
>>> from
>>> classpath so maybe LocallyResolvingServiceItem (not entirely keen on that
>>> but explains what's happening).
>>>
>>> On some level there's a bit of me that screams we have a broken design if
>>> we
>>> can't get decent names.....
>>>
>>> On 3 February 2011 09:10, Peter Firmstone <ji...@zeus.net.au> wrote:
>>>
>>>
>>>
>>>       
>>>> Dan Creswell wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> "In summary MarshalledServiceItem is a class that only resolves local
>>>>> code.
>>>>>  Interestingly MarshalledServiceItem, might not be an appropriate name,
>>>>> perhaps LocalServiceItem to indicate that only local code is utilised,
>>>>> on
>>>>> this reasoning, I can't see any objection to reflective proxy's being
>>>>> non
>>>>> null either."
>>>>>
>>>>> Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
>>>>> what's going on.....
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> The method unmarshall, sort of makes sense, can you think of a better
>>>> name?
>>>>
>>>>
>>>>
>>>>         
>>>
>>>       
>>     
>
>   

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

Thing is the ServiceItem is not the thing being resolved....

On 3 February 2011 09:59, Peter Firmstone <ji...@zeus.net.au> wrote:

> How about LocallyResolvedServiceItem?
>
>
> Dan Creswell wrote:
>
>> Sure, the method does indeed unmarshall but within specific constraints
>> and
>> those constraints are defined by the enclosing class (an instance of
>> ServiceItem in this case).
>>
>> So I'm thinking we're really talking about is a ServiceItem that pulls
>> from
>> classpath so maybe LocallyResolvingServiceItem (not entirely keen on that
>> but explains what's happening).
>>
>> On some level there's a bit of me that screams we have a broken design if
>> we
>> can't get decent names.....
>>
>> On 3 February 2011 09:10, Peter Firmstone <ji...@zeus.net.au> wrote:
>>
>>
>>
>>> Dan Creswell wrote:
>>>
>>>
>>>
>>>> "In summary MarshalledServiceItem is a class that only resolves local
>>>> code.
>>>>  Interestingly MarshalledServiceItem, might not be an appropriate name,
>>>> perhaps LocalServiceItem to indicate that only local code is utilised,
>>>> on
>>>> this reasoning, I can't see any objection to reflective proxy's being
>>>> non
>>>> null either."
>>>>
>>>> Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
>>>> what's going on.....
>>>>
>>>>
>>>>
>>>>
>>>>
>>> The method unmarshall, sort of makes sense, can you think of a better
>>> name?
>>>
>>>
>>>
>>
>>
>>
>
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

How about LocallyResolvedServiceItem?

Dan Creswell wrote:
> Sure, the method does indeed unmarshall but within specific constraints and
> those constraints are defined by the enclosing class (an instance of
> ServiceItem in this case).
>
> So I'm thinking we're really talking about is a ServiceItem that pulls from
> classpath so maybe LocallyResolvingServiceItem (not entirely keen on that
> but explains what's happening).
>
> On some level there's a bit of me that screams we have a broken design if we
> can't get decent names.....
>
> On 3 February 2011 09:10, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> Dan Creswell wrote:
>>
>>     
>>> "In summary MarshalledServiceItem is a class that only resolves local
>>> code.
>>>  Interestingly MarshalledServiceItem, might not be an appropriate name,
>>> perhaps LocalServiceItem to indicate that only local code is utilised, on
>>> this reasoning, I can't see any objection to reflective proxy's being non
>>> null either."
>>>
>>> Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
>>> what's going on.....
>>>
>>>
>>>
>>>       
>> The method unmarshall, sort of makes sense, can you think of a better name?
>>
>>     
>
>   

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

Sure, the method does indeed unmarshall but within specific constraints and
those constraints are defined by the enclosing class (an instance of
ServiceItem in this case).

So I'm thinking we're really talking about is a ServiceItem that pulls from
classpath so maybe LocallyResolvingServiceItem (not entirely keen on that
but explains what's happening).

On some level there's a bit of me that screams we have a broken design if we
can't get decent names.....

On 3 February 2011 09:10, Peter Firmstone <ji...@zeus.net.au> wrote:

> Dan Creswell wrote:
>
>> "In summary MarshalledServiceItem is a class that only resolves local
>> code.
>>  Interestingly MarshalledServiceItem, might not be an appropriate name,
>> perhaps LocalServiceItem to indicate that only local code is utilised, on
>> this reasoning, I can't see any objection to reflective proxy's being non
>> null either."
>>
>> Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
>> what's going on.....
>>
>>
>>
> The method unmarshall, sort of makes sense, can you think of a better name?
>

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Dan Creswell wrote:
> "In summary MarshalledServiceItem is a class that only resolves local code.
>  Interestingly MarshalledServiceItem, might not be an appropriate name,
> perhaps LocalServiceItem to indicate that only local code is utilised, on
> this reasoning, I can't see any objection to reflective proxy's being non
> null either."
>
> Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
> what's going on.....
>
>   
The method unmarshall, sort of makes sense, can you think of a better name?

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

"In summary MarshalledServiceItem is a class that only resolves local code.
 Interestingly MarshalledServiceItem, might not be an appropriate name,
perhaps LocalServiceItem to indicate that only local code is utilised, on
this reasoning, I can't see any objection to reflective proxy's being non
null either."

Indeed, the name MarshalledServiceItem is completely "out-of-whack" with
what's going on.....

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Gregg Wonderly wrote:
> On 2/2/2011 6:15 PM, Peter Firmstone wrote:
>> Michał Kłeczek wrote:
>>> On Thu, Feb 3, 2011 at 12:47 AM, Peter Firmstone <ji...@zeus.net.au> 
>>> wrote:
>>>> I like the fact that Gregg's pushed the envelope, he's pushed Jini 
>>>> beyond
>>>> it's current capability.
>>>>
>>>> Now I'm wondering if there's a way around the Preferred Classes 
>>>> problem.
>>>
>>> The problem with PREFERRED.LIST is more fundamental - for example
>>> every RemoteEvent producer is forced to download a jar file just to
>>> check that RemoteEventListener is not preferred. As such the issue
>>> cannot be solved at the LUS API level.
>>>
>>> Michal
>>>
>> Yes there are more fundamental problems with preferred classes, 
>> ClassLoading etc
>> that remain to be solved.
>>
>> But rather than fix it on this occassion, MarshalledServiceItem is 
>> just resolved
>> with local classes and preferred class loading is not used or needed. 
>> When the
>> client unmarshalls the MarshalledServiceItem, then preferred class 
>> loading will
>> be used and jar files downloaded. MarshalledServiceItem can provide a
>> significant amount of information by just resolving using local 
>> classes, which
>> the client can utilise.
>>
>> In the long term I see it like this:
>>
>> We need to look into dependency inversion to remove implementation 
>> classes from
>> the platform and application class loader, which is the problem that 
>> preferred
>> class loading addresses.
>>
>> If implementation classes are not present in the platform, you don't 
>> need
>> preferred classes for proxys to have their own copy of an 
>> implementation class,
>> because those classes will be resolved by standard java classloading 
>> rules in
>> the proxy class loader.
>
> One thing to note, is that I have used PreferredClassLoader to fix 
> bugs by taking common classes that I use in GUI constructions that 
> would have been resolved in the parent loader and switching them to be 
> preferred so that I'd get the fixed version.  I am not sure that there 
> is a perfect "split" of implementation vs platform.  Sometimes 
> implementation can be a mini-platform of sorts.  Just 
> repackaging/rearranging jars is not a singular fix, always.
>
> Gregg Wonderly
>

Based on our discussions, I've made changes to MarshalledServiceItem and 
StreamServiceRegistrar (see below), I have reduced 
StreamServiceRegistrar to a single lookup method, my reasoning is, the 
other methods, based on those already existing in ServiceRegistrar, are 
there primarily to build a suitable ServiceTemplate to avoid downloading 
proxy's, however since we're providing an alternate means 
(MarshalledServiceItem) to avoid or delay downloading proxy's those 3 
methods are less relevant.

So what we have is a new lookup service interface for the internet, that 
extends ServiceRegistrar and provides one new method, with means to 
provide Entry's that resolve locally for querying without downloading 
any jar files, until such time as the client wants to, this could also 
avoid downloading the MarshalledInstance of the proxy until 
unmarshalling is performed.

In summary MarshalledServiceItem is a class that only resolves local 
code.  Interestingly MarshalledServiceItem, might not be an appropriate 
name, perhaps LocalServiceItem to indicate that only local code is 
utilised, on this reasoning, I can't see any objection to reflective 
proxy's being non null either.

What ramifications does this have for proxy trust if we already trust 
the lookup service?

MarshalledServiceItem's single method, unmarshall, is not declared to 
throw an IOException or remote exception, for the case where a 
ServiceItem cannot be unmarshalled, fields are set to null, as is the 
current practice for ServiceItem.  Does anyone think this method should 
declare any exceptions?

Cheers,

Peter.


/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.apache.river.api.lookup;

import java.net.URI;
import java.security.CodeSource;
import net.jini.core.entry.Entry;
import net.jini.core.lookup.ServiceID;
import net.jini.core.lookup.ServiceItem;

/**
 * MarshalledServiceItem extends ServiceItem and can be used anywhere a
 * ServiceItem can.  A MarshalledServiceItem implementation instance
 * contains the marshalled form of a Service and it's Entry's,
 * the corresponding superclass ServiceItem however contains null values
 * for the service and can exclude any Entry's, however where Entry
 * classes already exist at the client, that they be unmarshalled.
 *
 * The ServiceID shall be in unmarshalled form always in the ServiceItem 
super class.
 *
 * Since the ServiceItem.service is null, use of this class in existing 
software
 * will not return the service, however it will not break that software as
 * ServiceItem's contract is to set service or Entry's to null when they 
cannot
 * be unmarshalled.
 *
 * ServiceItem's toString() method will return a different result for
 * MarshalledServiceItem instances.
 *
 * If required, a new ServiceItem that is fully unmarshalled
 * can be constructed from this class's methods and ServiceID.
 *
 * @author Peter Firmstone.
 */
public abstract class MarshalledServiceItem extends ServiceItem{
    private static final long SerialVersionUID = 1L;
    protected MarshalledServiceItem(ServiceID id, Entry[] 
unmarshalledEntries){
        super(id, (Object) null, unmarshalledEntries);
    }
   
    /* Default constructor for serializable sub class.
     */
    protected MarshalledServiceItem(){
        super(null, null, null);
    }
    /**
     * Unmarshall the ServiceItem.  The returned ServiceItem may have a 
null service
     * or entries , if unmarshalling is unsuccessful.
     *
     * @return ServiceItem, totally unmarshalled.
     */
    public abstract ServiceItem unmarshall();
}



/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership. The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License. You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.apache.river.api.lookup;

import java.io.IOException;
import net.jini.core.lookup.ServiceRegistrar;
import net.jini.core.lookup.ServiceTemplate;
import net.jini.core.lookup.ServiceItem;
import org.apache.river.api.util.ResultStream;

/**
 * Defines the interface to the lookup service.  The interface is not a
 * remote interface; each implementation of the lookup service exports
 * proxy objects that implement the StreamServiceRegistrar interface 
local to
 * the client, using an implementation-specific protocol to communicate
 * with the actual remote server.  All of the proxy methods obey normal
 * RMI remote interface semantics except where explicitly noted.  Two
 * proxy objects are equal if they are proxies for the same lookup service.
 * Every method invocation (on both StreamServiceRegistrar and 
ServiceRegistration)
 * is atomic with respect to other invocations.
 *
 * The StreamServiceRegistrar is intended to perform the same function
 * as the ServiceRegistrar, but with the ability to return results as a
 * stream, so memory consumption can be minimised at the client.
 *
 * All clients utilising ServiceRegistrar, should switch to the
 * StreamServiceRegistrar.
 *
 * @see ServiceRegistrar
 * @see PortableServiceRegistrar
 * @see ServiceRegistration
 * @author Peter Firmstone
 * @since 2.2.0
 */
public interface StreamServiceRegistrar extends ServiceRegistrar{

    /**
     * Returns a ResultStream that provides access to MarshalledServiceItem
     * instances.  The ResultStream terminates with a null value.  The 
result
     * stream may be infinite.
     *
     * MarshalledServiceItem extends ServiceItem and can be used anywhere a
     * ServiceItem can.  A MarshalledServiceItem implementation instance
     * contains the marshalled form of a Service and it's Entry's,
     * the corresponding superclass ServiceItem however contains null values
     * for the service and excludes any Entry's that are not 
specifically requested
     * unmarshalled by this method. The ServiceID will be unmarshalled 
always.
     *
     * This method is designed to allow the caller to control exactly what
     * is unmarshalled and when, it allows unmarshalling of specific entries
     * that the caller may wish to utilise for filtering. It is
     * designed to allow both the caller and the implementer to deal 
with very
     * large result sets in an incremental fashion.
     *
     * It is absolutely essential that the caller closes and deletes any 
references to
     * the returned result stream as soon as it is no longer requried.
     *
     * @param tmpl template to match
     * specified template
     * @param unmarshalledEntries only Entry's with these classes will be in
     * unmarshalled form.
     * @param maxBatchSize Allows the caller to limit the number of results
     * held locally, larger batch sizes reduce network traffic, but may 
delay
     * processing locally depending on implementation.
     * @return ResultStream containing ServiceItem's
     * @throws java.rmi.RemoteException
     * @see MarshalledServiceItem
     * @see ServiceItem
     * @see ResultStream
     * @see ServiceResultStreamFilter
     * @see ResultStreamUnmarshaller
     * @since 2.2.0
     */
    ResultStream lookup(ServiceTemplate tmpl,
        Class[] unmarshalledEntries, int maxBatchSize) throws IOException;
}

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

On 2/2/2011 6:15 PM, Peter Firmstone wrote:
> Michał Kłeczek wrote:
>> On Thu, Feb 3, 2011 at 12:47 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>>> I like the fact that Gregg's pushed the envelope, he's pushed Jini beyond
>>> it's current capability.
>>>
>>> Now I'm wondering if there's a way around the Preferred Classes problem.
>>
>> The problem with PREFERRED.LIST is more fundamental - for example
>> every RemoteEvent producer is forced to download a jar file just to
>> check that RemoteEventListener is not preferred. As such the issue
>> cannot be solved at the LUS API level.
>>
>> Michal
>>
> Yes there are more fundamental problems with preferred classes, ClassLoading etc
> that remain to be solved.
>
> But rather than fix it on this occassion, MarshalledServiceItem is just resolved
> with local classes and preferred class loading is not used or needed. When the
> client unmarshalls the MarshalledServiceItem, then preferred class loading will
> be used and jar files downloaded. MarshalledServiceItem can provide a
> significant amount of information by just resolving using local classes, which
> the client can utilise.
>
> In the long term I see it like this:
>
> We need to look into dependency inversion to remove implementation classes from
> the platform and application class loader, which is the problem that preferred
> class loading addresses.
>
> If implementation classes are not present in the platform, you don't need
> preferred classes for proxys to have their own copy of an implementation class,
> because those classes will be resolved by standard java classloading rules in
> the proxy class loader.

One thing to note, is that I have used PreferredClassLoader to fix bugs by 
taking common classes that I use in GUI constructions that would have been 
resolved in the parent loader and switching them to be preferred so that I'd get 
the fixed version.  I am not sure that there is a perfect "split" of 
implementation vs platform.  Sometimes implementation can be a mini-platform of 
sorts.  Just repackaging/rearranging jars is not a singular fix, always.

Gregg Wonderly

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Michał Kłeczek wrote:
> On Thu, Feb 3, 2011 at 12:47 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>   
>> I like the fact that Gregg's pushed the envelope, he's pushed Jini beyond
>> it's current capability.
>>
>> Now I'm wondering if there's a way around the Preferred Classes problem.
>>     
>
> The problem with PREFERRED.LIST is more fundamental - for example
> every RemoteEvent producer is forced to download a jar file just to
> check that RemoteEventListener is not preferred. As such the issue
> cannot be solved at the LUS API level.
>
> Michal
>
>   
Yes there are more fundamental problems with preferred classes, 
ClassLoading etc that remain to be solved.

But rather than fix it on this occassion, MarshalledServiceItem is just 
resolved with local classes and preferred class loading is not used or 
needed. When the client unmarshalls the MarshalledServiceItem, then 
preferred class loading will be used and jar files downloaded. 
MarshalledServiceItem can provide a significant amount of information by 
just resolving using local classes, which the client can utilise.

In the long term I see it like this:

We need to look into dependency inversion to remove implementation 
classes from the platform and application class loader, which is the 
problem that preferred class loading addresses.

If implementation classes are not present in the platform, you don't 
need preferred classes for proxys to have their own copy of an 
implementation class, because those classes will be resolved by standard 
java classloading rules in the proxy class loader.

Cheers,

Peter.

Re: MarshalledServiceItem

[Michał Kłeczek <mi...@xpro.biz>]

On Thu, Feb 3, 2011 at 12:47 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> I like the fact that Gregg's pushed the envelope, he's pushed Jini beyond
> it's current capability.
>
> Now I'm wondering if there's a way around the Preferred Classes problem.

The problem with PREFERRED.LIST is more fundamental - for example
every RemoteEvent producer is forced to download a jar file just to
check that RemoteEventListener is not preferred. As such the issue
cannot be solved at the LUS API level.

Michal

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

I like the fact that Gregg's pushed the envelope, he's pushed Jini 
beyond it's current capability.

Now I'm wondering if there's a way around the Preferred Classes problem.

In Reggie's proxy implementation, during registration, fields in Entry 
classes, if they can be resolved to standard java classes, are, however 
they are sent as a MarshalledWrapper (wrapping a MarshalledInstance), if 
they can't.

For MarshalledServiceItem, we don't have to honour preferred classes, 
since we can unmarshall everything correctly when the unmarshall method 
is called, in other words we can unmarshall the Entry's again, using 
preferred class loading and downloaded codebases.

For implementation of MarshalledServiceItem, the reggie implementation 
EntryClass can be first used to identify an entry we're interested in, 
first we check if the name matches an Entry's Class name we're 
interested in, secondly we resolve all fields of these Entry's locally 
by providing a get() method on MarshalledInstance that ignores codebase 
annotations, leaving any field that cannot be unresolved as null.

So what we get at the client is a number of Entry's that may be fully 
unmarshalled, or partly so, which should give us enough information to 
determine if the service is interesting.

When the client finds a field in an Entry in the ServiceItem, set to 
null, and it wants to know more, it must then unmarshall the 
MarshalledServiceItem, a process which then retrieves all codebases and 
correctly unmarshalls all Entry's and the Service, the original array of 
Entry's contained in the MarshalledServiceItem is discarded, so the fact 
that these were previously resolved locally doesn't matter.

Then we could leave Preferred classes alone.

Thoughts?

Peter.

Gregg Wonderly wrote:
> I am finding my self being slightly defensive sounding and I'm not 
> wanting to be defensive about my implementation.  I want to defend the 
> features that I need and thus made changes in lookup to provide.  
> Hopefully the inline below is not degenerating into to much 
> reiteration of the issues.  There are some additional thoughts about 
> other ways that lookup could be changed to provide the "string" 
> information without a proxy service.
>
> See below...
>
> On 2/2/2011 3:26 AM, Dan Creswell wrote:
>> Inline and leaving context for now.....
>>
>> On 1 February 2011 19:12, Gregg Wonderly<ge...@cox.net>  wrote:
>>
>>> On 2/1/2011 3:17 AM, Dan Creswell wrote:
>>>
>>>> So...
>>>>
>>>> On 1 February 2011 01:29, Gregg Wonderly<gr...@wonderly.org>   wrote:
>>>>
>>>>   One of the important things for my use of my changes to the
>>>>> ServiceRegistrar interface was separating deserialization of the 
>>>>> Entrys
>>>>> from
>>>>> the service object.
>>>>>
>>>>
>>> Note "separating" used here.  I was trying to say that I need to be 
>>> able to
>>> "pick" which Entry objects to deserialize, and deserialize them 
>>> independent
>>> of deserializing the service proxy/object.
>>>
>>>
>>>   Being able to defer downloading was also controlled by the changes to
>>>>> ClassLoading to include the "neverPrefer" settings on a class 
>>>>> name.  I
>>>>> can
>>>>> do that now through the recent RMIClassLoaderSPI override 
>>>>> capabilities.
>>>>>   But
>>>>> I still need to be able to control which Entry is deserialized.  I 
>>>>> need
>>>>> to
>>>>> be able to ask what classes, by name, are in the namespace of each 
>>>>> Entry
>>>>> object as my implementation allowed. I had made changes to Reggie
>>>>> marshalling to use reflection to get the class hierarchy, and then
>>>>> provided
>>>>> access to the list of classnames as part of the returned, marshalled
>>>>> value
>>>>> in my API.
>>>>>
>>>>>
>>>>>   Digging through classes belonging to a service implementation 
>>>>> that is
>>>> designed to be encapsulated/not your concern? Does that sound right? I
>>>> gotta
>>>> say, it sounds horribly invasive, very hacky and speaks of an overly
>>>> complex
>>>> solution as the result of treating symptoms, not problems.
>>>>
>>>
>>> Okay, I want to go through the problems that I was dealing with, the
>>> attributes of the current mechanisms, and how I inserted points of 
>>> control
>>> to manage these issues.
>>>
>>> First, let's look at the simple client mechanisms I wanted to have.  
>>> I have
>>> a desktop environment, which does lookup, and then shows treeviews of
>>> services based on "machine"->"group"->Service Instance structure.  The
>>> deployed services contain Entry objects that provide values for 
>>> these three
>>> name spaces.  Since I need to look at these, they can not be 
>>> preferred by
>>> the service or I can't read them.  There may be a subclass that the 
>>> client
>>> has used, and that can be preferred no problem.  There are some 
>>> other items
>>> in the Entrys that I need too, such as icons.  Any of the Entry 
>>> objects that
>>> I need access to, and any classes which they are dependent on, and 
>>> which are
>>> publicly visible classes/interfaces, because of that attribute 
>>> (public) can
>>> not be realistically preferred.
>>>
>>>
>> These Entry types are part of the basic infrastructure of your system 
>> aren't
>> they? Why download them at all? I guess because you have to "teach" 
>> it to
>> the LUS via code-download? Or you could just put them on all classpaths
>> including LUS? S'far as I can see you've got an extended LUS 
>> infrastructure
>> that everyone shares thus codebase seems like the wrong choice for 
>> these.
>>
>> Sure user's can subclass Entry types but do they? Do you even want 
>> them to?
>> Why? Thus far I've seen many subclass Entry's for JavaSpaces and, I 
>> think,
>> zero for LUS type annotations.
>>
>> See my other thread with Peter but it seems to me that you have some 
>> core
>> additions to LUS infrastructure and a need to be able to filter what you
>> don't want to see/is outside of a client's knowledge base. Because 
>> it's that
>> inability to filter that then leads to lots of additional downloading.
>
> The basic issue is that I need to show users what services exist.  The 
> LUS maintains a very useful view of the health of the network.  Asking 
> it what services exist has to happen by some 'client'.  Remember that 
> my customers techs are using the client out in the field, on a slow 
> network.  So whether they enumerate something from the LUS or from 
> another "service", there will be network traffic.
>
> It might be possible to enumerate services at a aggregation service, 
> but then you have to unmarshall all of the services there to get the 
> "data" that you'd send to a client.  I don't want to have 100s of 
> proxies active on the network for this mechanism, really.  The 
> deferred unmarshalling in discussion with Peter would help deal with 
> this by allowing such an aggregation service to not have to unmarshall 
> the proxy.  That was my initial intent.  It was only after I got that 
> implemented that I found out how the PREFERRED.LIST really kept it 
> from working at all, and so I had to do something to keep the 
> PREFERRED.LIST from being accessed when not needed yet.
>
> The service aggregation becomes a single point of failure and I'd need 
> to replicate it to every server that had an LUS on it.  Then, it 
> really becomes more like a secondary look up service, and so I'd still 
> have to ask, is the LUS API really in the right form if there is a 
> "mirror" of it needed?  Note that I made it possible to pick through a 
> lot of the class information for the Entry and the service Proxy in 
> reef's work because I was trying to get around having to mess with the 
> never-preferred business.  But in the end, I just could not find any 
> other way, because as soon as I wanted to unmarshall an Entry to get a 
> "Name" and a "Group" and an "Icon" for my UI, PreferredClassLoading 
> went after the PREFERRED.LIST, even though not a single class that I 
> was using would be resolved from the codebase.
>
> I'm not sure how I would "not activate the codebase" in some client.  
> It needs the Entry object, fully resolved.  I want the codebase to 
> work right, so I'm not gonna just throw in a proxy context class 
> loader that resolves everything locally.  If there is content in an 
> Entry that is an unknown class, I will suffer the download to make 
> sure that it resolves to the right codebase.
>
> Note that I am not wanting to focus on the never-prefer issue as 
> something that I need.  What I actually need is to defer download of 
> any part of the codebase until the service is "activated".
>
> What might be interesting is to figure out how to stuff the preferred 
> list into the MarshalledObject annotation so that PreferredClassLoader 
> would not need to retrieve the codebase elements to resolve how to 
> perform class loading.
>
> Currently, there is a lot of backward compatibility involved in how 
> the annotation is managed.
>
> The namespace change that we are discussing for com.sun.jini, might be 
> the right moment to introduce any compatibility issues with 
> MarshalledObject and the use of the RMIClassLoader annotation.
>
>>> So my first assertion is that Entry objects will hardly ever be 
>>> preferred,
>>> and thus downloading of the PREFERRED.LIST is not a necessary step to
>>> correctly resolve them for my use.
>>>
>>> The next issues is this PREFERRED.LIST file.  The implementation of 
>>> that
>>> technology is that the file is retrieved from the first jar file 
>>> listed in
>>> the codebase.  So, in order to resolve any class in the proxy or Entry
>>> values, there must be a download from the associated codebase server.
>>>
>>> Now, one of the first things that I think about with distributed 
>>> computing
>>> is the business of partial failure and limiting that impact to the 
>>> overall
>>> performance of the system.  My deployments at my customers' sites 
>>> consist of
>>> copies of the same service platform with the same services on it.  
>>> There are
>>> copies for both redundancy and for capacity.
>>>
>>> Each instance of my platform has as a starting point 20 some 
>>> services each
>>> with a separate codebase, but with many common jars such as 
>>> jsk-dk.jar.  If
>>> a lookup service responds with a list of services but the codebase 
>>> server
>>> doesn't respond with jars, then any attempt to download limits the
>>> performance of the display of available services, and I considered an
>>> "incorrect list of available services" less important than 
>>> non-display or
>>> slow-display of available services.
>>
>> As Michal says elsewhere, are you sure the solution is to try and 
>> leverage
>> LUS to handle display challenges like this?
>
> I need the data with minimal network traffic.  The LUS lookup is even 
> something that generates some delays.  But, the UI, as it populates, 
> feels largely like a web page load.  I've reached a point of 
> acceptable performance that works for now.
>
> Service lookup and selection has always been a point of contention as 
> you well know.  I had always been on the side of the fence that said 
> the LUS works fine the way it is.  It wasn't until I tried to figure 
> out how to manage 100s of services visible and only 1 being 
> interesting that I suddenly found myself setting on the fence asking, 
> "If it almost does the right thing, is there a way to maintain the 
> same interface, but add a new interface with more functionality that 
> would keep me from having to completely ignore the LUS?"
>
> So, I tried something out.  I find that given how Reggie works and is 
> implemented, that the addition of the functionality as shown in Reef 
> does provide some new opportunities.  But, delayed unmarshalling 
> alone, does not provide for no-download until needed.  
> PreferredClassLoading is still in the mix.  My solution to that, which 
> I think does need to be revisited.  We need to decide if the 
> PREFERRED.LIST is really a problem because it causes a download. 
> Service registration data will always require download, and so just to 
> use the Entry values in a UI or some other client will still require 
> download of all the elements of the codebase needed to resolve the 
> classes.
>
> In one explanation of how things are currently working with River, 
> without my changes, I suggested that using Reggie and preferred class 
> loading would be like opening up a Google session, running a query, 
> and then having to download every matched pages content before you 
> could see the list of search results.
>
> Google provides the type of service that you describe by changing what 
> the user has to experience to be something that is usable.  But we 
> don't all then go and write a new service on top of google.  It does 
> what is needed to find something to look at.  But, it still then has 
> the struggle that you have to click on a link, and wait for the page 
> to load in order to validate which content is appropriate to your 
> needs.  The textual summaries don't always provide the right "view" of 
> what you are looking for, even through they've tried to provide a 
> textual view of the document fragment that contains what you searched 
> for.
>
> It is the load-before-need-is-established behavior that I am trying to 
> raise the red flag about.  We need a complete different LUS if there 
> are more than a handful of services that match a lookup request.
>
> I feel really strongly against forcing clients to resolve 10s of 
> services much less 100s just to see what they might choose to use.  
> Entry and service interface based filtering works to trim results 
> dramatically.  But in the case of a ServiceUI discovery where you are 
> just looking for services that have a
>
> Service registration with a ServiceRegistrar implementation is the "I 
> am alive" moment, and I think we need to maintain that single point of 
> information because doing otherwise spreads knowledge out into more 
> than one place and makes for a distributed database that I don't feel 
> we don't need to add to the mix.
>
>> What you seem to be doing is constructing a navigation structure for 
>> a GUI
>> direct from the LUS per client. Why wouldn't you have some separate 
>> "service
>> summary service" that builds up this navigation structure by tracking
>> service arrivals and such and creating the necessary descriptive text 
>> for
>> clients to pull down in one big bang?
>
> There is some tangible benefits I can see in doing this, but I don't 
> like the idea that this is another layer between the client and the 
> LUS regarding availability and liveness.  As I said above, the LUS has 
> the information, but it's the "unmarshalling" (ServiceRegistrar's 
> APIs) and PreferredClassLoading that don't allow a client of the LUS 
> to avoid downloading what they don't need to download.  That's why I 
> went specifically at those two sources.
>
> If I didn't have the sources to mess with, I am sure I would have had 
> a different approach or done something without Jini even.
>
>> You get caching for staters, no need to download a bunch of services 
>> until
>> later and thus far fewer downloads especially as things like Entry's 
>> can be
>> rendered into text reps as part of this process of building navigation
>> information. (heck you could even insist all Entry's provide a 
>> toString for
>> use on your platform). No reason too why the summary service couldn't 
>> build
>> on demand trees for some specified set of Entry types. It could cache 
>> those
>> and update them too such that if another client requests the same 
>> tree it
>> gets it straight back.
>>
>> I guess my summary would be, I feel like you've gone for a "deep" 
>> solution
>> that builds into existing infrastructure instead of building 
>> extensions atop
>> that specifically tackle your problems in a more "meta" fashion. Please
>> don't take offence at that, I'm not for a moment saying it's a 
>> mistake, just
>> one path to solution and given what I've heard so far I think there's
>> another path that hasn't been ruled out yet.
>
> I'm good with this discussion.  That never happened before, even at 
> the Jini Community meeting and later when I pushed Reef out onto 
> java.net.
>
> I want to really stress what I feel are the primary issues for me and 
> my customers.  I also feel that given the larger set of services that 
> might be visible on a broader network environment, such as the 
> internet, that these issue will be important to others as well.  It 
> sounds like Chris at least did the caching of jars.
>
> Imagine if ServiceRegistrar included an XML or other string document 
> that could be used by applications to get "client display" 
> information, instead of just Entry object values, and that 
> PreferredClassLoader got the PREFERRED.LIST from the annotation String 
> instead of from a META-INF entry in the jar file.
>
> Then, the ServiceRegistrar could still be a primary source of service 
> registration and lookup but would include some flexibility for local 
> needs.  We could even cause the XML to be constructed by the 
> serialization process so that it looked something like:
>
> <entries>
>     <entry class="net.jini.lookup.entry.Name">
>         <field name="name" type="java.lang.String" value="My Service"/>
>     </entry>
> </entries>
>
> All the public fields would be enumerated and specified with values 
> that would allow complete use of them (when they have appropriate 
> toString() values) and thus we can get the text from the Entry objects 
> directly.
> `
> We'd just provide an API that provided the programmatic iteration over 
> this information without XML being directly visible.
>
> So the existing lookup would be fine, and ServiceItem could have an 
> additional XML string field to pass to a new API.
>
> Just thinking out loud here...
>
> Gregg
>

Re: MarshalledServiceItem

[Gregg Wonderly <gr...@wonderly.org>]

I am finding my self being slightly defensive sounding and I'm not wanting to be 
defensive about my implementation.  I want to defend the features that I need 
and thus made changes in lookup to provide.  Hopefully the inline below is not 
degenerating into to much reiteration of the issues.  There are some additional 
thoughts about other ways that lookup could be changed to provide the "string" 
information without a proxy service.

See below...

On 2/2/2011 3:26 AM, Dan Creswell wrote:
> Inline and leaving context for now.....
>
> On 1 February 2011 19:12, Gregg Wonderly<ge...@cox.net>  wrote:
>
>> On 2/1/2011 3:17 AM, Dan Creswell wrote:
>>
>>> So...
>>>
>>> On 1 February 2011 01:29, Gregg Wonderly<gr...@wonderly.org>   wrote:
>>>
>>>   One of the important things for my use of my changes to the
>>>> ServiceRegistrar interface was separating deserialization of the Entrys
>>>> from
>>>> the service object.
>>>>
>>>
>> Note "separating" used here.  I was trying to say that I need to be able to
>> "pick" which Entry objects to deserialize, and deserialize them independent
>> of deserializing the service proxy/object.
>>
>>
>>   Being able to defer downloading was also controlled by the changes to
>>>> ClassLoading to include the "neverPrefer" settings on a class name.  I
>>>> can
>>>> do that now through the recent RMIClassLoaderSPI override capabilities.
>>>>   But
>>>> I still need to be able to control which Entry is deserialized.  I need
>>>> to
>>>> be able to ask what classes, by name, are in the namespace of each Entry
>>>> object as my implementation allowed. I had made changes to Reggie
>>>> marshalling to use reflection to get the class hierarchy, and then
>>>> provided
>>>> access to the list of classnames as part of the returned, marshalled
>>>> value
>>>> in my API.
>>>>
>>>>
>>>>   Digging through classes belonging to a service implementation that is
>>> designed to be encapsulated/not your concern? Does that sound right? I
>>> gotta
>>> say, it sounds horribly invasive, very hacky and speaks of an overly
>>> complex
>>> solution as the result of treating symptoms, not problems.
>>>
>>
>> Okay, I want to go through the problems that I was dealing with, the
>> attributes of the current mechanisms, and how I inserted points of control
>> to manage these issues.
>>
>> First, let's look at the simple client mechanisms I wanted to have.  I have
>> a desktop environment, which does lookup, and then shows treeviews of
>> services based on "machine"->"group"->Service Instance structure.  The
>> deployed services contain Entry objects that provide values for these three
>> name spaces.  Since I need to look at these, they can not be preferred by
>> the service or I can't read them.  There may be a subclass that the client
>> has used, and that can be preferred no problem.  There are some other items
>> in the Entrys that I need too, such as icons.  Any of the Entry objects that
>> I need access to, and any classes which they are dependent on, and which are
>> publicly visible classes/interfaces, because of that attribute (public) can
>> not be realistically preferred.
>>
>>
> These Entry types are part of the basic infrastructure of your system aren't
> they? Why download them at all? I guess because you have to "teach" it to
> the LUS via code-download? Or you could just put them on all classpaths
> including LUS? S'far as I can see you've got an extended LUS infrastructure
> that everyone shares thus codebase seems like the wrong choice for these.
>
> Sure user's can subclass Entry types but do they? Do you even want them to?
> Why? Thus far I've seen many subclass Entry's for JavaSpaces and, I think,
> zero for LUS type annotations.
>
> See my other thread with Peter but it seems to me that you have some core
> additions to LUS infrastructure and a need to be able to filter what you
> don't want to see/is outside of a client's knowledge base. Because it's that
> inability to filter that then leads to lots of additional downloading.

The basic issue is that I need to show users what services exist.  The LUS 
maintains a very useful view of the health of the network.  Asking it what 
services exist has to happen by some 'client'.  Remember that my customers techs 
are using the client out in the field, on a slow network.  So whether they 
enumerate something from the LUS or from another "service", there will be 
network traffic.

It might be possible to enumerate services at a aggregation service, but then 
you have to unmarshall all of the services there to get the "data" that you'd 
send to a client.  I don't want to have 100s of proxies active on the network 
for this mechanism, really.  The deferred unmarshalling in discussion with Peter 
would help deal with this by allowing such an aggregation service to not have to 
unmarshall the proxy.  That was my initial intent.  It was only after I got that 
implemented that I found out how the PREFERRED.LIST really kept it from working 
at all, and so I had to do something to keep the PREFERRED.LIST from being 
accessed when not needed yet.

The service aggregation becomes a single point of failure and I'd need to 
replicate it to every server that had an LUS on it.  Then, it really becomes 
more like a secondary look up service, and so I'd still have to ask, is the LUS 
API really in the right form if there is a "mirror" of it needed?  Note that I 
made it possible to pick through a lot of the class information for the Entry 
and the service Proxy in reef's work because I was trying to get around having 
to mess with the never-preferred business.  But in the end, I just could not 
find any other way, because as soon as I wanted to unmarshall an Entry to get a 
"Name" and a "Group" and an "Icon" for my UI, PreferredClassLoading went after 
the PREFERRED.LIST, even though not a single class that I was using would be 
resolved from the codebase.

I'm not sure how I would "not activate the codebase" in some client.  It needs 
the Entry object, fully resolved.  I want the codebase to work right, so I'm not 
gonna just throw in a proxy context class loader that resolves everything 
locally.  If there is content in an Entry that is an unknown class, I will 
suffer the download to make sure that it resolves to the right codebase.

Note that I am not wanting to focus on the never-prefer issue as something that 
I need.  What I actually need is to defer download of any part of the codebase 
until the service is "activated".

What might be interesting is to figure out how to stuff the preferred list into 
the MarshalledObject annotation so that PreferredClassLoader would not need to 
retrieve the codebase elements to resolve how to perform class loading.

Currently, there is a lot of backward compatibility involved in how the 
annotation is managed.

The namespace change that we are discussing for com.sun.jini, might be the right 
moment to introduce any compatibility issues with MarshalledObject and the use 
of the RMIClassLoader annotation.

>> So my first assertion is that Entry objects will hardly ever be preferred,
>> and thus downloading of the PREFERRED.LIST is not a necessary step to
>> correctly resolve them for my use.
>>
>> The next issues is this PREFERRED.LIST file.  The implementation of that
>> technology is that the file is retrieved from the first jar file listed in
>> the codebase.  So, in order to resolve any class in the proxy or Entry
>> values, there must be a download from the associated codebase server.
>>
>> Now, one of the first things that I think about with distributed computing
>> is the business of partial failure and limiting that impact to the overall
>> performance of the system.  My deployments at my customers' sites consist of
>> copies of the same service platform with the same services on it.  There are
>> copies for both redundancy and for capacity.
>>
>> Each instance of my platform has as a starting point 20 some services each
>> with a separate codebase, but with many common jars such as jsk-dk.jar.  If
>> a lookup service responds with a list of services but the codebase server
>> doesn't respond with jars, then any attempt to download limits the
>> performance of the display of available services, and I considered an
>> "incorrect list of available services" less important than non-display or
>> slow-display of available services.
>
> As Michal says elsewhere, are you sure the solution is to try and leverage
> LUS to handle display challenges like this?

I need the data with minimal network traffic.  The LUS lookup is even something 
that generates some delays.  But, the UI, as it populates, feels largely like a 
web page load.  I've reached a point of acceptable performance that works for now.

Service lookup and selection has always been a point of contention as you well 
know.  I had always been on the side of the fence that said the LUS works fine 
the way it is.  It wasn't until I tried to figure out how to manage 100s of 
services visible and only 1 being interesting that I suddenly found myself 
setting on the fence asking, "If it almost does the right thing, is there a way 
to maintain the same interface, but add a new interface with more functionality 
that would keep me from having to completely ignore the LUS?"

So, I tried something out.  I find that given how Reggie works and is 
implemented, that the addition of the functionality as shown in Reef does 
provide some new opportunities.  But, delayed unmarshalling alone, does not 
provide for no-download until needed.  PreferredClassLoading is still in the 
mix.  My solution to that, which I think does need to be revisited.  We need to 
decide if the PREFERRED.LIST is really a problem because it causes a download. 
Service registration data will always require download, and so just to use the 
Entry values in a UI or some other client will still require download of all the 
elements of the codebase needed to resolve the classes.

In one explanation of how things are currently working with River, without my 
changes, I suggested that using Reggie and preferred class loading would be like 
opening up a Google session, running a query, and then having to download every 
matched pages content before you could see the list of search results.

Google provides the type of service that you describe by changing what the user 
has to experience to be something that is usable.  But we don't all then go and 
write a new service on top of google.  It does what is needed to find something 
to look at.  But, it still then has the struggle that you have to click on a 
link, and wait for the page to load in order to validate which content is 
appropriate to your needs.  The textual summaries don't always provide the right 
"view" of what you are looking for, even through they've tried to provide a 
textual view of the document fragment that contains what you searched for.

It is the load-before-need-is-established behavior that I am trying to raise the 
red flag about.  We need a complete different LUS if there are more than a 
handful of services that match a lookup request.

I feel really strongly against forcing clients to resolve 10s of services much 
less 100s just to see what they might choose to use.  Entry and service 
interface based filtering works to trim results dramatically.  But in the case 
of a ServiceUI discovery where you are just looking for services that have a

Service registration with a ServiceRegistrar implementation is the "I am alive" 
moment, and I think we need to maintain that single point of information because 
doing otherwise spreads knowledge out into more than one place and makes for a 
distributed database that I don't feel we don't need to add to the mix.

> What you seem to be doing is constructing a navigation structure for a GUI
> direct from the LUS per client. Why wouldn't you have some separate "service
> summary service" that builds up this navigation structure by tracking
> service arrivals and such and creating the necessary descriptive text for
> clients to pull down in one big bang?

There is some tangible benefits I can see in doing this, but I don't like the 
idea that this is another layer between the client and the LUS regarding 
availability and liveness.  As I said above, the LUS has the information, but 
it's the "unmarshalling" (ServiceRegistrar's APIs) and PreferredClassLoading 
that don't allow a client of the LUS to avoid downloading what they don't need 
to download.  That's why I went specifically at those two sources.

If I didn't have the sources to mess with, I am sure I would have had a 
different approach or done something without Jini even.

> You get caching for staters, no need to download a bunch of services until
> later and thus far fewer downloads especially as things like Entry's can be
> rendered into text reps as part of this process of building navigation
> information. (heck you could even insist all Entry's provide a toString for
> use on your platform). No reason too why the summary service couldn't build
> on demand trees for some specified set of Entry types. It could cache those
> and update them too such that if another client requests the same tree it
> gets it straight back.
>
> I guess my summary would be, I feel like you've gone for a "deep" solution
> that builds into existing infrastructure instead of building extensions atop
> that specifically tackle your problems in a more "meta" fashion. Please
> don't take offence at that, I'm not for a moment saying it's a mistake, just
> one path to solution and given what I've heard so far I think there's
> another path that hasn't been ruled out yet.

I'm good with this discussion.  That never happened before, even at the Jini 
Community meeting and later when I pushed Reef out onto java.net.

I want to really stress what I feel are the primary issues for me and my 
customers.  I also feel that given the larger set of services that might be 
visible on a broader network environment, such as the internet, that these issue 
will be important to others as well.  It sounds like Chris at least did the 
caching of jars.

Imagine if ServiceRegistrar included an XML or other string document that could 
be used by applications to get "client display" information, instead of just 
Entry object values, and that PreferredClassLoader got the PREFERRED.LIST from 
the annotation String instead of from a META-INF entry in the jar file.

Then, the ServiceRegistrar could still be a primary source of service 
registration and lookup but would include some flexibility for local needs.  We 
could even cause the XML to be constructed by the serialization process so that 
it looked something like:

<entries>
	<entry class="net.jini.lookup.entry.Name">
		<field name="name" type="java.lang.String" value="My Service"/>
	</entry>
</entries>

All the public fields would be enumerated and specified with values that would 
allow complete use of them (when they have appropriate toString() values) and 
thus we can get the text from the Entry objects directly.
`
We'd just provide an API that provided the programmatic iteration over this 
information without XML being directly visible.

So the existing lookup would be fine, and ServiceItem could have an additional 
XML string field to pass to a new API.

Just thinking out loud here...

Gregg

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

Inline and leaving context for now.....

On 1 February 2011 19:12, Gregg Wonderly <ge...@cox.net> wrote:

> On 2/1/2011 3:17 AM, Dan Creswell wrote:
>
>> So...
>>
>> On 1 February 2011 01:29, Gregg Wonderly<gr...@wonderly.org>  wrote:
>>
>>  One of the important things for my use of my changes to the
>>> ServiceRegistrar interface was separating deserialization of the Entrys
>>> from
>>> the service object.
>>>
>>
> Note "separating" used here.  I was trying to say that I need to be able to
> "pick" which Entry objects to deserialize, and deserialize them independent
> of deserializing the service proxy/object.
>
>
>  Being able to defer downloading was also controlled by the changes to
>>> ClassLoading to include the "neverPrefer" settings on a class name.  I
>>> can
>>> do that now through the recent RMIClassLoaderSPI override capabilities.
>>>  But
>>> I still need to be able to control which Entry is deserialized.  I need
>>> to
>>> be able to ask what classes, by name, are in the namespace of each Entry
>>> object as my implementation allowed. I had made changes to Reggie
>>> marshalling to use reflection to get the class hierarchy, and then
>>> provided
>>> access to the list of classnames as part of the returned, marshalled
>>> value
>>> in my API.
>>>
>>>
>>>  Digging through classes belonging to a service implementation that is
>> designed to be encapsulated/not your concern? Does that sound right? I
>> gotta
>> say, it sounds horribly invasive, very hacky and speaks of an overly
>> complex
>> solution as the result of treating symptoms, not problems.
>>
>
> Okay, I want to go through the problems that I was dealing with, the
> attributes of the current mechanisms, and how I inserted points of control
> to manage these issues.
>
> First, let's look at the simple client mechanisms I wanted to have.  I have
> a desktop environment, which does lookup, and then shows treeviews of
> services based on "machine"->"group"->Service Instance structure.  The
> deployed services contain Entry objects that provide values for these three
> name spaces.  Since I need to look at these, they can not be preferred by
> the service or I can't read them.  There may be a subclass that the client
> has used, and that can be preferred no problem.  There are some other items
> in the Entrys that I need too, such as icons.  Any of the Entry objects that
> I need access to, and any classes which they are dependent on, and which are
> publicly visible classes/interfaces, because of that attribute (public) can
> not be realistically preferred.
>
>
These Entry types are part of the basic infrastructure of your system aren't
they? Why download them at all? I guess because you have to "teach" it to
the LUS via code-download? Or you could just put them on all classpaths
including LUS? S'far as I can see you've got an extended LUS infrastructure
that everyone shares thus codebase seems like the wrong choice for these.

Sure user's can subclass Entry types but do they? Do you even want them to?
Why? Thus far I've seen many subclass Entry's for JavaSpaces and, I think,
zero for LUS type annotations.

See my other thread with Peter but it seems to me that you have some core
additions to LUS infrastructure and a need to be able to filter what you
don't want to see/is outside of a client's knowledge base. Because it's that
inability to filter that then leads to lots of additional downloading.



> So my first assertion is that Entry objects will hardly ever be preferred,
> and thus downloading of the PREFERRED.LIST is not a necessary step to
> correctly resolve them for my use.
>
> The next issues is this PREFERRED.LIST file.  The implementation of that
> technology is that the file is retrieved from the first jar file listed in
> the codebase.  So, in order to resolve any class in the proxy or Entry
> values, there must be a download from the associated codebase server.
>
> Now, one of the first things that I think about with distributed computing
> is the business of partial failure and limiting that impact to the overall
> performance of the system.  My deployments at my customers' sites consist of
> copies of the same service platform with the same services on it.  There are
> copies for both redundancy and for capacity.
>
> Each instance of my platform has as a starting point 20 some services each
> with a separate codebase, but with many common jars such as jsk-dk.jar.  If
> a lookup service responds with a list of services but the codebase server
> doesn't respond with jars, then any attempt to download limits the
> performance of the display of available services, and I considered an
> "incorrect list of available services" less important than non-display or
> slow-display of available services.
>

As Michal says elsewhere, are you sure the solution is to try and leverage
LUS to handle display challenges like this?

What you seem to be doing is constructing a navigation structure for a GUI
direct from the LUS per client. Why wouldn't you have some separate "service
summary service" that builds up this navigation structure by tracking
service arrivals and such and creating the necessary descriptive text for
clients to pull down in one big bang?

You get caching for staters, no need to download a bunch of services until
later and thus far fewer downloads especially as things like Entry's can be
rendered into text reps as part of this process of building navigation
information. (heck you could even insist all Entry's provide a toString for
use on your platform). No reason too why the summary service couldn't build
on demand trees for some specified set of Entry types. It could cache those
and update them too such that if another client requests the same tree it
gets it straight back.

I guess my summary would be, I feel like you've gone for a "deep" solution
that builds into existing infrastructure instead of building extensions atop
that specifically tackle your problems in a more "meta" fashion. Please
don't take offence at that, I'm not for a moment saying it's a mistake, just
one path to solution and given what I've heard so far I think there's
another path that hasn't been ruled out yet.


>
> The technicians in my customers' business have a lot of windshield time and
> use their cellular connectivity or satellite based WAN networking for many
> tasks. This means that there is a lot of latency, cost for bandwidth in term
> of time and bandwidth both.  So, for me to make a distributed network based
> services environment work, I have to limit network to only what is
> absolutely necessary.
>
> So, back to the two different changes that I made to service lookup and
> class resolution.  The activities in ClassLoading to provide a
> "neverPreferred" functionality was instituted by having my client tell
> ClassLoading via public static API, which Entry classes and dependent
> classes should never be preferred.  Recall, that in the beginning, the class
> loading mechanisms did have a "neverPreferred" policy for java.lang.* and
> for native types and arrays.
>
> This was removed in an update later on in 2.1 if I remember right.  This
> kept classloading from being triggered in those cases.  That functionality,
> when removed, was declared as nonessential/unused because normal class
> loading would keep that from happening.  However, because
> PreferredClassLoading will never ask the parent class loader (especially
> without my patch to allow RMIClassLoaderSPI to not be used) to resolve a
> class until the PREFERRED.LIST is on hand, the codebase will always be
> accessed, requiring a download of at least the first jar.
>
> In my case, a customer with 100 codebases (5 machines) trying to use my
> desktop client, would experience 100 http connections and downloads of the
> first jar before they could see all the available services.  And, because of
> the fact that not ever single codebase would be requested in parallel, any
> stuck codebase could potentially delay the display of available services for
> 3-5 minutes per problem codebase while TCP connect requests timeout.
>
> For me, this was just not an acceptable outcome.  It was simply not
> possible to manage my product environment with these issues at hand.  No
> matter how much the first jar required to be transferred, it would simply
> take way too long for them to see a useful client display.
>
> My changes in River-336 for RMIClassLoaderSPI override, use methods such as
> CodebaseAccessClassLoader.loadClass(...) which can be used to provide the
> "neverPreferred" functionality.  So, I have a pluggable mechanism that
> allows me to manage this issue.  So if no one else has such requirements,
> there is no need to discuss this as a point of argument for inclusion in
> River.  I think it is a valid point of control because the client can
> declare the platform that they are compatible with by using this facility to
> never prefer certain classes which will then cause services overrides to not
> appear and the client could even provide some feedback when this happens by
> allow the CodebaseAccessClassLoader implementation to provide some data
> points about when it finds it is ignoring something declared as preferred.
>
> The other business at hand is the deferred unmarshalling.  And in that
> case, I don't think that there is too much of an argument to be had.  If we
> focus on just allowing the client to access the entries selectively, and
> defer unmarshalling of the service proxy/object separately, that's the
> basics of what I need for my environment.
>
> The API that I implemented allowed access to the list of classes and
> interfaces referred to by the Entry so that the client could discover that
> it can't unmarshall the Entry because it requires network access to download
> something that it doesn't have locally.  If that usage pattern doesn't seem
> valuable, then something more along the lines of providing a Class object as
> Peter's API shows might work well enough.  But we need to figure out how to
> deal with Entry values which are subclasses.  If I pass in Name.class, and
> there are several subclasses of Name visible in the Entry values, then which
> one would you return?  All of them?  What if one of those contained a rather
> large object graph which was not what I wanted.  Would I get it unmarshalled
> so that I could still inspect it to decide if it was the one I wanted to
> unmarshall and look at?
>
> Sorry for going on and on about this, but I wanted to try and put out all
> the details that I had dealt with and the thoughts around my path of
> solution.
>
> Gregg Wonderly
>
>

Re: MarshalledServiceItem

[Michal Kleczek <kl...@gmail.com>]

On Wed, 2011-02-02 at 18:07 +1000, Peter Firmstone wrote:
> >> First, let's look at the simple client mechanisms I wanted to have.  I have a 
> >> desktop environment, which does lookup, and then shows treeviews of services 
> >> based on "machine"->"group"->Service Instance structure.  The deployed services 
> >> contain Entry objects that provide values for these three name spaces.  Since I 
> >> need to look at these, they can not be preferred by the service or I can't read 
> >> them.  There may be a subclass that the client has used, and that can be 
> >> preferred no problem.  There are some other items in the Entrys that I need too, 
> >> such as icons.  Any of the Entry objects that I need access to, and any classes 
> >> which they are dependent on, and which are publicly visible classes/interfaces, 
> >> because of that attribute (public) can not be realistically preferred.
> >>     
> >
> > I think I'm with Dan here - looks to me like a misuse of LUS. I see it
> > as a more or less dumb bootstrap mechanism - designed for _lookup_ not
> > _browsing_ .
> > Wouldn't a ServiceBrowserService (or a ServiceIndexingService) be a
> > solution?
> >
> Can you give an example?
>

import javax.swing.tree.TreeModel;

//the service is deployed in a Djinn - it keeps track of all available
//services and indexes them so that the client can just use a TreeModel
//to display available services
interface ServiceBrowserService {
  TreeModel getServiceTree();
}

Don't really know what example you need - the point is that the problems
Greg described don't really have to be dealt with by the client but by
one of the services in Djinn that would provide clients with a _view_ of
all available services.

Paraphrasing:
"All problems in a Djinn can be solved by another service".

Michal

Re: MarshalledServiceItem

[Peter Firmstone <ji...@zeus.net.au>]

Michal Kleczek wrote:
> On Tue, 2011-02-01 at 13:12 -0600, Gregg Wonderly wrote:
>   
>>> Digging through classes belonging to a service implementation that is
>>> designed to be encapsulated/not your concern? Does that sound right? I gotta
>>> say, it sounds horribly invasive, very hacky and speaks of an overly complex
>>> solution as the result of treating symptoms, not problems.
>>>       
>> Okay, I want to go through the problems that I was dealing with, the attributes 
>> of the current mechanisms, and how I inserted points of control to manage these 
>> issues.
>>
>> First, let's look at the simple client mechanisms I wanted to have.  I have a 
>> desktop environment, which does lookup, and then shows treeviews of services 
>> based on "machine"->"group"->Service Instance structure.  The deployed services 
>> contain Entry objects that provide values for these three name spaces.  Since I 
>> need to look at these, they can not be preferred by the service or I can't read 
>> them.  There may be a subclass that the client has used, and that can be 
>> preferred no problem.  There are some other items in the Entrys that I need too, 
>> such as icons.  Any of the Entry objects that I need access to, and any classes 
>> which they are dependent on, and which are publicly visible classes/interfaces, 
>> because of that attribute (public) can not be realistically preferred.
>>     
>
> I think I'm with Dan here - looks to me like a misuse of LUS. I see it
> as a more or less dumb bootstrap mechanism - designed for _lookup_ not
> _browsing_ .
> Wouldn't a ServiceBrowserService (or a ServiceIndexingService) be a
> solution?
>
> Michal
>
>   

Can you give an example?

Peter.

Re: MarshalledServiceItem

[Michal Kleczek <mi...@xpro.biz>]

On Tue, 2011-02-01 at 13:12 -0600, Gregg Wonderly wrote:
> > Digging through classes belonging to a service implementation that is
> > designed to be encapsulated/not your concern? Does that sound right? I gotta
> > say, it sounds horribly invasive, very hacky and speaks of an overly complex
> > solution as the result of treating symptoms, not problems.
> 
> Okay, I want to go through the problems that I was dealing with, the attributes 
> of the current mechanisms, and how I inserted points of control to manage these 
> issues.
> 
> First, let's look at the simple client mechanisms I wanted to have.  I have a 
> desktop environment, which does lookup, and then shows treeviews of services 
> based on "machine"->"group"->Service Instance structure.  The deployed services 
> contain Entry objects that provide values for these three name spaces.  Since I 
> need to look at these, they can not be preferred by the service or I can't read 
> them.  There may be a subclass that the client has used, and that can be 
> preferred no problem.  There are some other items in the Entrys that I need too, 
> such as icons.  Any of the Entry objects that I need access to, and any classes 
> which they are dependent on, and which are publicly visible classes/interfaces, 
> because of that attribute (public) can not be realistically preferred.

I think I'm with Dan here - looks to me like a misuse of LUS. I see it
as a more or less dumb bootstrap mechanism - designed for _lookup_ not
_browsing_ .
Wouldn't a ServiceBrowserService (or a ServiceIndexingService) be a
solution?

Michal

Re: MarshalledServiceItem

[Gregg Wonderly <ge...@cox.net>]

On 2/1/2011 3:17 AM, Dan Creswell wrote:
> So...
>
> On 1 February 2011 01:29, Gregg Wonderly<gr...@wonderly.org>  wrote:
>
>> One of the important things for my use of my changes to the
>> ServiceRegistrar interface was separating deserialization of the Entrys from
>> the service object.

Note "separating" used here.  I was trying to say that I need to be able to 
"pick" which Entry objects to deserialize, and deserialize them independent of 
deserializing the service proxy/object.

>> Being able to defer downloading was also controlled by the changes to
>> ClassLoading to include the "neverPrefer" settings on a class name.  I can
>> do that now through the recent RMIClassLoaderSPI override capabilities.  But
>> I still need to be able to control which Entry is deserialized.  I need to
>> be able to ask what classes, by name, are in the namespace of each Entry
>> object as my implementation allowed. I had made changes to Reggie
>> marshalling to use reflection to get the class hierarchy, and then provided
>> access to the list of classnames as part of the returned, marshalled value
>> in my API.
>>
>>
> Digging through classes belonging to a service implementation that is
> designed to be encapsulated/not your concern? Does that sound right? I gotta
> say, it sounds horribly invasive, very hacky and speaks of an overly complex
> solution as the result of treating symptoms, not problems.

Okay, I want to go through the problems that I was dealing with, the attributes 
of the current mechanisms, and how I inserted points of control to manage these 
issues.

First, let's look at the simple client mechanisms I wanted to have.  I have a 
desktop environment, which does lookup, and then shows treeviews of services 
based on "machine"->"group"->Service Instance structure.  The deployed services 
contain Entry objects that provide values for these three name spaces.  Since I 
need to look at these, they can not be preferred by the service or I can't read 
them.  There may be a subclass that the client has used, and that can be 
preferred no problem.  There are some other items in the Entrys that I need too, 
such as icons.  Any of the Entry objects that I need access to, and any classes 
which they are dependent on, and which are publicly visible classes/interfaces, 
because of that attribute (public) can not be realistically preferred.

So my first assertion is that Entry objects will hardly ever be preferred, and 
thus downloading of the PREFERRED.LIST is not a necessary step to correctly 
resolve them for my use.

The next issues is this PREFERRED.LIST file.  The implementation of that 
technology is that the file is retrieved from the first jar file listed in the 
codebase.  So, in order to resolve any class in the proxy or Entry values, there 
must be a download from the associated codebase server.

Now, one of the first things that I think about with distributed computing is 
the business of partial failure and limiting that impact to the overall 
performance of the system.  My deployments at my customers' sites consist of 
copies of the same service platform with the same services on it.  There are 
copies for both redundancy and for capacity.

Each instance of my platform has as a starting point 20 some services each with 
a separate codebase, but with many common jars such as jsk-dk.jar.  If a lookup 
service responds with a list of services but the codebase server doesn't respond 
with jars, then any attempt to download limits the performance of the display of 
available services, and I considered an "incorrect list of available services" 
less important than non-display or slow-display of available services.

The technicians in my customers' business have a lot of windshield time and use 
their cellular connectivity or satellite based WAN networking for many tasks. 
This means that there is a lot of latency, cost for bandwidth in term of time 
and bandwidth both.  So, for me to make a distributed network based services 
environment work, I have to limit network to only what is absolutely necessary.

So, back to the two different changes that I made to service lookup and class 
resolution.  The activities in ClassLoading to provide a "neverPreferred" 
functionality was instituted by having my client tell ClassLoading via public 
static API, which Entry classes and dependent classes should never be preferred. 
  Recall, that in the beginning, the class loading mechanisms did have a 
"neverPreferred" policy for java.lang.* and for native types and arrays.

This was removed in an update later on in 2.1 if I remember right.  This kept 
classloading from being triggered in those cases.  That functionality, when 
removed, was declared as nonessential/unused because normal class loading would 
keep that from happening.  However, because PreferredClassLoading will never ask 
the parent class loader (especially without my patch to allow RMIClassLoaderSPI 
to not be used) to resolve a class until the PREFERRED.LIST is on hand, the 
codebase will always be accessed, requiring a download of at least the first jar.

In my case, a customer with 100 codebases (5 machines) trying to use my desktop 
client, would experience 100 http connections and downloads of the first jar 
before they could see all the available services.  And, because of the fact that 
not ever single codebase would be requested in parallel, any stuck codebase 
could potentially delay the display of available services for 3-5 minutes per 
problem codebase while TCP connect requests timeout.

For me, this was just not an acceptable outcome.  It was simply not possible to 
manage my product environment with these issues at hand.  No matter how much the 
first jar required to be transferred, it would simply take way too long for them 
to see a useful client display.

My changes in River-336 for RMIClassLoaderSPI override, use methods such as 
CodebaseAccessClassLoader.loadClass(...) which can be used to provide the 
"neverPreferred" functionality.  So, I have a pluggable mechanism that allows me 
to manage this issue.  So if no one else has such requirements, there is no need 
to discuss this as a point of argument for inclusion in River.  I think it is a 
valid point of control because the client can declare the platform that they are 
compatible with by using this facility to never prefer certain classes which 
will then cause services overrides to not appear and the client could even 
provide some feedback when this happens by allow the CodebaseAccessClassLoader 
implementation to provide some data points about when it finds it is ignoring 
something declared as preferred.

The other business at hand is the deferred unmarshalling.  And in that case, I 
don't think that there is too much of an argument to be had.  If we focus on 
just allowing the client to access the entries selectively, and defer 
unmarshalling of the service proxy/object separately, that's the basics of what 
I need for my environment.

The API that I implemented allowed access to the list of classes and interfaces 
referred to by the Entry so that the client could discover that it can't 
unmarshall the Entry because it requires network access to download something 
that it doesn't have locally.  If that usage pattern doesn't seem valuable, then 
something more along the lines of providing a Class object as Peter's API shows 
might work well enough.  But we need to figure out how to deal with Entry values 
which are subclasses.  If I pass in Name.class, and there are several subclasses 
of Name visible in the Entry values, then which one would you return?  All of 
them?  What if one of those contained a rather large object graph which was not 
what I wanted.  Would I get it unmarshalled so that I could still inspect it to 
decide if it was the one I wanted to unmarshall and look at?

Sorry for going on and on about this, but I wanted to try and put out all the 
details that I had dealt with and the thoughts around my path of solution.

Gregg Wonderly

Re: MarshalledServiceItem

[Dan Creswell <da...@gmail.com>]

So...

On 1 February 2011 01:29, Gregg Wonderly <gr...@wonderly.org> wrote:

> One of the important things for my use of my changes to the
> ServiceRegistrar interface was separating deserialization of the Entrys from
> the service object.  Ultimately, I wanted to not have to worry about how the
> "codebase" was structured in order to minimize downloading. Some Jini
> deployers have done things to put a small jar at the front of the codebase
> that just has the preferred.list in it.  Even that download, for me, would
> have been too much.  My customers' deployments have nearly 50 codebases
> visible when my client starts up.  Those 50 connections over a cellular or
> other high latency network would make service selection take way too long.
>
>
"deserialization of Entry's from the service object"? What does "service
object" refer to?

I think you _have_ to worry about how the codebase is structured. Anything
else is a hack and if we're going to do hacks there are better ones like
bundling up people's .jars for them etc so save on connections/add caching
and such. Doing it further downstream in the lookup mechanism is treating a
symptom not the problem. Not the sort of thing that keeps a design clean or
consistent.



Being able to defer downloading was also controlled by the changes to
> ClassLoading to include the "neverPrefer" settings on a class name.  I can
> do that now through the recent RMIClassLoaderSPI override capabilities.  But
> I still need to be able to control which Entry is deserialized.  I need to
> be able to ask what classes, by name, are in the namespace of each Entry
> object as my implementation allowed. I had made changes to Reggie
> marshalling to use reflection to get the class hierarchy, and then provided
> access to the list of classnames as part of the returned, marshalled value
> in my API.
>
>
Digging through classes belonging to a service implementation that is
designed to be encapsulated/not your concern? Does that sound right? I gotta
say, it sounds horribly invasive, very hacky and speaks of an overly complex
solution as the result of treating symptoms, not problems.

Which brings me to a point Peter made elsewhere: "delayed umarshalling and
selective unmarshalling of Entry's are very important features, missing from
River."

Important to whom? How many people are in the category?

If it's a lot of people, we should spend some time evaluating whether the
default class resolution/download/lookup approach is the generic norm and
potential replacements.

Simplistically, delaying downloading is as simple as returning a
MarshalledInstance on which one can then call .get as and when one
desires.....

Now, I haven't had my coffee yet and I've got a pile of operational worries
to attend to this morning which means I'm maybe a little shout'y,
nevertheless I for one am not convinced of the value of this and less
convinced we have the right solution which feels complex, exceptional and
clunky.


Gregg Wonderly
>
> On Jan 30, 2011, at 2:23 PM, Peter Firmstone wrote:
>
> > Gut feel, tells me the following class is not right, it's intent is to
> provide an api for implementation of delayed unmarshalling and provisioning
> of codebases, for services implementing a lookup service or
> ServiceRegistrar, whilst retaining backward compatibility.
> >
> > In other words, the lookup service proxy would implement it.
> >
> > Thoughts I had were to declare IOException's on methods, and to return a
> String annotation, rather than a URI.
> >
> > What are your thoughts?
> >
> > Cheers,
> >
> > Peter.
> >
> > /*
> > * Licensed to the Apache Software Foundation (ASF) under one
> > * or more contributor license agreements.  See the NOTICE file
> > * distributed with this work for additional information
> > * regarding copyright ownership. The ASF licenses this file
> > * to you under the Apache License, Version 2.0 (the
> > * "License"); you may not use this file except in compliance
> > * with the License. You may obtain a copy of the License at
> > *
> > *      http://www.apache.org/licenses/LICENSE-2.0
> > *
> > * Unless required by applicable law or agreed to in writing, software
> > * distributed under the License is distributed on an "AS IS" BASIS,
> > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> > * See the License for the specific language governing permissions and
> > * limitations under the License.
> > */
> >
> > package org.apache.river.api.lookup;
> >
> > import java.net.URI;
> > import java.security.CodeSource;
> > import net.jini.core.entry.Entry;
> > import net.jini.core.lookup.ServiceID;
> > import net.jini.core.lookup.ServiceItem;
> >
> > /**
> > * MarshalledServiceItem extends ServiceItem and can be used anywhere a
> > * ServiceItem can.  A MarshalledServiceItem implementation instance
> > * contains the marshalled form of a Service and it's Entry's,
> > * the corresponding superclass ServiceItem however contains null values
> > * for the service and can exclude any Entry's, however where Entry
> > * classes already exist at the client, that they be unmarshalled.
> > *
> > * The ServiceID shall be in unmarshalled form always in the ServiceItem
> super class.
> > *
> > * Since the ServiceItem.service is null, use of this class in existing
> software
> > * will not return the service, however it will not break that software as
> > * ServiceItem's contract is to set service or Entry's to null when they
> cannot
> > * be unmarshalled.
> > *
> > * ServiceItem's toString() method will return a different result for
> > * MarshalledServiceItem instances.
> > *
> > * If required, a new ServiceItem that is fully unmarshalled
> > * can be constructed from this class's methods and ServiceID.
> > *
> > * @author Peter Firmstone.
> > */
> > public abstract class MarshalledServiceItem extends ServiceItem{
> >   private static final long SerialVersionUID = 1L;
> >   protected MarshalledServiceItem(ServiceID id, Entry[]
> unmarshalledEntries){
> >       super(id, (Object) null, unmarshalledEntries);
> >   }
> >   /**
> >    * Unmarshall the service proxy.
> >    * @param load service with local or existing CodeSource or null for
> >    * default.
> >    * @return the service proxy, null if class not found.
> >    */
> >   public abstract Object getService(CodeSource[] code);
> >   /**
> >    * Unmarshall the Entry's
> >    * @return array of Entry's, null entry in array for any class not
> found.
> >    */
> >   public abstract Entry[] getEntries();
> >     public abstract URI[] getAnnotations();
> > }
> >
>
>