You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@datafu.apache.org by "jian wang (JIRA)" <ji...@apache.org> on 2014/02/11 14:13:19 UTC
[jira] [Comment Edited] (DATAFU-16) weighted reservoir sampling
with exponential jumps UDF
[ https://issues.apache.org/jira/browse/DATAFU-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897828#comment-13897828 ]
jian wang edited comment on DATAFU-16 at 2/11/14 1:11 PM:
----------------------------------------------------------
I have updated the WeightedSamplingCorrectnessTests.java and there is a simulated perf test within. Following is the ouptut of the test.
[testng] *** Running reservoirExpJPerfTest ***
[testng] Output:
[testng] accumulateDuration accumulateExpJDuration
[testng] 8563 1563
accumulateDuration: test duration for weighted sampling without exp jump in accumulate mode
accumulateExpJDuration: test duration for weighted sampling with exp jump
unit is milliseconds.
The 2 algos are conducted under the same input.
was (Author: king821221):
I have updated the WeightedSamplingCorrectnessTests.java and there is a simulated perf test within. Following is the ouptut of the test.
[testng] *** Running reservoirExpJPerfTest ***
[testng] Output:
[testng] accumulateDuration accumulateExpJDuration
[testng] 8563 1563
accumulateDuration: test duration for weighted sampling without exp jump in accumulate mode
accumulateExpJDuration: test duration for weighted sampling with exp jump
unit is milliseconds
> weighted reservoir sampling with exponential jumps UDF
> ------------------------------------------------------
>
> Key: DATAFU-16
> URL: https://issues.apache.org/jira/browse/DATAFU-16
> Project: DataFu
> Issue Type: New Feature
> Environment: Mac, Linux
> pig-0.11
> Reporter: jian wang
> Priority: Minor
> Attachments: ScoredExpJmpReservoir.java, ScoredReservoir.java, WeightedSamplingCorrectnessTests.java
>
>
> Create a weightedReservoirSampleWithExpJump UDF to implement the weighted reservoir sampling algorithm with exponential jumps. Investigation is tracked in https://github.com/linkedin/datafu/issues/80. This task is part of experiment of different weighted sampling algorithms.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)