You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pirk.apache.org by tellison <gi...@git.apache.org> on 2016/08/12 15:08:08 UTC
[GitHub] incubator-pirk pull request #57: [PIRK-49] PrimitivePartioner does not accou...
GitHub user tellison opened a pull request:
https://github.com/apache/incubator-pirk/pull/57
[PIRK-49] PrimitivePartioner does not account for locale settings
- Ensure bytes are always encoded in network byte ordering.
- Remove primitive type partitioner's unhealthy obsession with byte buffers.
- Simplify getPaddedPartitions method.
- Enhanced tests for data partitioning.
- Further tidy up of concrete type to interfaces.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tellison/incubator-pirk partitioner
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-pirk/pull/57.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #57
----
commit 27d05ff2ea7b6343ff39274287e46e83c163c8c9
Author: Tim Ellison <t....@gmail.com>
Date: 2016-08-12T14:40:41Z
Enhancements to data partitioners.
- Ensure bytes are always encoded in network byte ordering.
- Remove primitive type partitioner's unhealthy obsession with byte
buffers.
- Simplify getPaddedPartitions method.
- Enhanced tests for data partitioning.
- Further tidy up of concrete type to interfaces.
commit 662ccbd1fd11545406762605f21abf6d02f840cc
Author: Tim Ellison <t....@gmail.com>
Date: 2016-08-12T14:44:06Z
Merge branch 'master' into partitioner
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #57: [PIRK-49] PrimitivePartioner does not accou...
Posted by smarthi <gi...@git.apache.org>.
Github user smarthi commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/57#discussion_r74619625
--- Diff: src/main/java/org/apache/pirk/schema/data/partitioner/PrimitiveTypePartitioner.java ---
@@ -346,53 +309,14 @@ public Object fromPartitions(List<BigInteger> parts, int partsIndex, String type
* Method to get an empty set of partitions by data type - used for padding return array values
*/
@Override
- public ArrayList<BigInteger> getPaddedPartitions(String type) throws PIRException
+ public List<BigInteger> getPaddedPartitions(String type) throws PIRException
{
- ArrayList<BigInteger> parts = new ArrayList<>();
-
int numParts = getNumPartitions(type);
- switch (type)
- {
- case BYTE:
- parts.add(new BigInteger(ByteBuffer.allocate(1).put(Byte.parseByte("0")).array()));
- break;
- case STRING:
- for (int i = 0; i < numParts; ++i)
- {
- parts.add(new BigInteger(ByteBuffer.allocate(1).put(Byte.parseByte("0")).array()));
- }
- break;
- default:
- // Extract the byte array
- byte[] bytes = new byte[0];
- switch (type)
- {
- case SHORT:
- bytes = ByteBuffer.allocate(numParts).putShort(Short.parseShort("0")).array();
- break;
- case INT:
- bytes = ByteBuffer.allocate(numParts).putInt(Integer.parseInt("0")).array();
- break;
- case LONG:
- bytes = ByteBuffer.allocate(numParts).putLong(Long.parseLong("0")).array();
- break;
- case FLOAT:
- bytes = ByteBuffer.allocate(numParts).putFloat(Float.parseFloat("0")).array();
- break;
- case DOUBLE:
- bytes = ByteBuffer.allocate(numParts).putDouble(Double.parseDouble("0")).array();
- break;
- case CHAR:
- bytes = ByteBuffer.allocate(numParts).putChar('0').array();
- break;
- }
- // Add bytes to parts ArrayList
- for (byte b : bytes)
- {
- parts.add(new BigInteger(ByteBuffer.allocate(1).put(b).array()));
- }
- break;
+ List<BigInteger> parts = new ArrayList<>(numParts);
+ for (int i = 0; i < numParts; i++)
+ {
+ parts.add(BigInteger.ZERO);
--- End diff --
How bout something like
`BigInteger[] bigInts = new BigInteger[numParts];
Arrays.fill(bigInts, BigInteger.ZERO);
List<BigInteger> parts = Arrays.asList(bigInts);`
that way, the for loop can be avoided in the code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk issue #57: [PIRK-49] PrimitivePartioner does not account for ...
Posted by ellisonanne <gi...@git.apache.org>.
Github user ellisonanne commented on the issue:
https://github.com/apache/incubator-pirk/pull/57
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #57: [PIRK-49] PrimitivePartioner does not accou...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-pirk/pull/57
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk issue #57: [PIRK-49] PrimitivePartioner does not account for ...
Posted by smarthi <gi...@git.apache.org>.
Github user smarthi commented on the issue:
https://github.com/apache/incubator-pirk/pull/57
lgtm +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---