You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Aran Donohue (JIRA)" <ji...@apache.org> on 2010/03/08 01:02:28 UTC
[jira] Created: (THRIFT-728) Make generated Haskell code an
instance of Arbitrary
Make generated Haskell code an instance of Arbitrary
----------------------------------------------------
Key: THRIFT-728
URL: https://issues.apache.org/jira/browse/THRIFT-728
Project: Thrift
Issue Type: New Feature
Components: Compiler (Haskell), Library (Haskell)
Environment: All
Reporter: Aran Donohue
Priority: Minor
Attachments: arbitrary.patch
The patch
* Generates Arbitrary instances for Thrift structs, enums and exceptions
* Provides Arbitrary instances for GHC's Int64, Data.Map.Map and Data.Set.Set
* Makes Thrift enums instances of Bounded and improves the Enum instance declaration
Making a type an instance of Test.QuickCheck.Arbitrary specifies how to generate random instances of the struct. This is useful for testing.
For example, consider the following simple Thrift declaration:
{code:title=distributed_log.thrift}
enum LogLevel { DEBUG, INFO, WARNING, ERROR }
typedef i32 UnixTimeStamp
struct LogEntry {
1: LogLevel level,
2: UnixTimeStamp timestamp,
3: string message
}
service Logger {
void log(1: LogEntry entry)
}
{code}
With the patch, the following program (import statements elided) is a fuzzer for the log service.
{code:title=LogTest.hs}
-- ripped from Test.QuickCheck.Gen.sample'
infexamples ∷ Gen a → IO [a]
infexamples (MkGen m) =
do rnd ← newStdGen
let rnds rnd = rnd1 : rnds rnd2 where (rnd1, rnd2) = split rnd
return [(m r n) | (r, n) ← rnds rnd `zip` [0,2..] ]
infentries = infexamples (arbitrary ∷ Gen LogEntry)
main = do entries ← infentries
forM entries logAnEntry
logAnEntry entry = do
transport ← hOpen ("localhost", PortNumber 9090)
let binProto = BinaryProtocol transport
let client = (binProto, binProto)
(Client.log client) entry
tClose transport
`Control.Exception.catch`
(λ(TransportExn s t) → print s)
--On systems with Haskell we could just generate a fuzzer like this one.
{code}
In implementing the Arbitrary instances, it was useful to make Thrift enums instances of Bounded and to improve the Enum instance. Specifically, whereas before,
{code}
[DEBUG .. ]
{code}
would throw an exception, now it behaves as expected without an exception.
I consider the patch incomplete. It's more of a starting point for a discussion at this point than a serious candidate for inclusion. If it is of interest, I'd appreciate some direction on testing it as well as style, and I'd welcome any other comments or thoughts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-728) Make generated Haskell code an
instance of Arbitrary
Posted by "Aran Donohue (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aran Donohue updated THRIFT-728:
--------------------------------
Attachment: (was: arbitrary.patch)
> Make generated Haskell code an instance of Arbitrary
> ----------------------------------------------------
>
> Key: THRIFT-728
> URL: https://issues.apache.org/jira/browse/THRIFT-728
> Project: Thrift
> Issue Type: New Feature
> Components: Compiler (Haskell), Library (Haskell)
> Environment: All
> Reporter: Aran Donohue
> Priority: Minor
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> The patch
> * Generates Arbitrary instances for Thrift structs, enums and exceptions
> * Provides Arbitrary instances for GHC's Int64, Data.Map.Map and Data.Set.Set
> * Makes Thrift enums instances of Bounded and improves the Enum instance declaration
> Making a type an instance of Test.QuickCheck.Arbitrary specifies how to generate random instances of the struct. This is useful for testing.
> For example, consider the following simple Thrift declaration:
> {code:title=distributed_log.thrift}
> enum LogLevel { DEBUG, INFO, WARNING, ERROR }
>
> typedef i32 UnixTimeStamp
>
> struct LogEntry {
> 1: LogLevel level,
> 2: UnixTimeStamp timestamp,
> 3: string message
> }
>
> service Logger {
> void log(1: LogEntry entry)
> }
> {code}
> With the patch, the following program (import statements elided) is a fuzzer for the log service.
> {code:title=LogTest.hs}
> -- ripped from Test.QuickCheck.Gen.sample'
> infexamples ∷ Gen a → IO [a]
> infexamples (MkGen m) =
> do rnd ← newStdGen
> let rnds rnd = rnd1 : rnds rnd2 where (rnd1, rnd2) = split rnd
> return [(m r n) | (r, n) ← rnds rnd `zip` [0,2..] ]
>
> infentries = infexamples (arbitrary ∷ Gen LogEntry)
>
> main = do entries ← infentries
> forM entries logAnEntry
>
> logAnEntry entry = do
> transport ← hOpen ("localhost", PortNumber 9090)
> let binProto = BinaryProtocol transport
> let client = (binProto, binProto)
> (Client.log client) entry
> tClose transport
> `Control.Exception.catch`
> (λ(TransportExn s t) → print s)
> --On systems with Haskell we could just generate a fuzzer like this one.
> {code}
> In implementing the Arbitrary instances, it was useful to make Thrift enums instances of Bounded and to improve the Enum instance. Specifically, whereas before,
> {code}
> [DEBUG .. ]
> {code}
> would throw an exception, now it behaves as expected without an exception.
> I consider the patch incomplete. It's more of a starting point for a discussion at this point than a serious candidate for inclusion. If it is of interest, I'd appreciate some direction on testing it as well as style, and I'd welcome any other comments or thoughts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-728) Make generated Haskell code an
instance of Arbitrary
Posted by "Iain Proctor (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885728#action_12885728 ]
Iain Proctor commented on THRIFT-728:
-------------------------------------
looks good
> Make generated Haskell code an instance of Arbitrary
> ----------------------------------------------------
>
> Key: THRIFT-728
> URL: https://issues.apache.org/jira/browse/THRIFT-728
> Project: Thrift
> Issue Type: New Feature
> Components: Compiler (Haskell), Library (Haskell)
> Environment: All
> Reporter: Aran Donohue
> Priority: Minor
> Attachments: arbitrary.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> The patch
> * Generates Arbitrary instances for Thrift structs, enums and exceptions
> * Provides Arbitrary instances for GHC's Int64, Data.Map.Map and Data.Set.Set
> * Makes Thrift enums instances of Bounded and improves the Enum instance declaration
> Making a type an instance of Test.QuickCheck.Arbitrary specifies how to generate random instances of the struct. This is useful for testing.
> For example, consider the following simple Thrift declaration:
> {code:title=distributed_log.thrift}
> enum LogLevel { DEBUG, INFO, WARNING, ERROR }
>
> typedef i32 UnixTimeStamp
>
> struct LogEntry {
> 1: LogLevel level,
> 2: UnixTimeStamp timestamp,
> 3: string message
> }
>
> service Logger {
> void log(1: LogEntry entry)
> }
> {code}
> With the patch, the following program (import statements elided) is a fuzzer for the log service.
> {code:title=LogTest.hs}
> -- ripped from Test.QuickCheck.Gen.sample'
> infexamples ∷ Gen a → IO [a]
> infexamples (MkGen m) =
> do rnd ← newStdGen
> let rnds rnd = rnd1 : rnds rnd2 where (rnd1, rnd2) = split rnd
> return [(m r n) | (r, n) ← rnds rnd `zip` [0,2..] ]
>
> infentries = infexamples (arbitrary ∷ Gen LogEntry)
>
> main = do entries ← infentries
> forM entries logAnEntry
>
> logAnEntry entry = do
> transport ← hOpen ("localhost", PortNumber 9090)
> let binProto = BinaryProtocol transport
> let client = (binProto, binProto)
> (Client.log client) entry
> tClose transport
> `Control.Exception.catch`
> (λ(TransportExn s t) → print s)
> --On systems with Haskell we could just generate a fuzzer like this one.
> {code}
> In implementing the Arbitrary instances, it was useful to make Thrift enums instances of Bounded and to improve the Enum instance. Specifically, whereas before,
> {code}
> [DEBUG .. ]
> {code}
> would throw an exception, now it behaves as expected without an exception.
> I consider the patch incomplete. It's more of a starting point for a discussion at this point than a serious candidate for inclusion. If it is of interest, I'd appreciate some direction on testing it as well as style, and I'd welcome any other comments or thoughts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
cthrift v.0.0.8
Posted by ma...@bestweb.net.
Hi, all!
release 0.0.8 of cthrift is now up at
https://sourceforge.net/projects/cthrift/files/cthrift-0.0.8.tgz/download.
Its more feature complete (oneway, map & set are the major missing
components. It can't support lists of lists either).
You may want to
have a quick look at the wiki, perhaps at the quick start section.
https://sourceforge.net/apps/mediawiki/cthrift/index.php?title=Quick_Start
Re: Performance of Thrift C++ client
Posted by ma...@bestweb.net.
FWIW:
I measured the performance of the client-side scribe
interface in cthrift for a few different data-points. I'm only reporting
the serialization times, not the time spent in the single write per
call.
list size 1 category="" msg="": 2755
ticks
list size 100 category="" msg="": 122489
ticks
list size 1 category="" msg=1000B: 2904
ticks
list size 100 category="" msg=1000B: 198899
ticks
I'm running a 2.6GHz Pentium dual-core E5300 with cygwin on
top of Vista Home Premium. All numbers are measured using RDTSC. Numbers
are the average of a 1000 message sends.
The scribe interface
is:
enum ResultCode{
OK,
TRY_LATER
}
struct LogEntry{
1: string category,
2: string message
}
ResultCode Log(1: list<LogEntry> messages),
> I don't have metrics on hand, but you almost always want to wrap
your base
> transport with a buffered or framed transport. At
least in my experience
> with Ruby and Java, a buffer makes all
the difference.
>
> -Bryan
>
> On Sun,
Mar 7, 2010 at 7:57 PM, <ma...@bestweb.net> wrote:
>
>>
>>
>>
>> Does anyone have any
performance metrics for Thrift generated
>> client-side code?
C++ would be nice.
>> As you might guess, I'm trying
>> to estimate the difference in the performance of the Thrift
generated
>> C++
>> code for TBinaryProtocol over
TSockets or TBufferedTransports and the C
>> code generated by
cthrift.
>>
>> Thanks!
>> Mayan
>>
>>
>>
>
Re: Performance of Thrift C++ client
Posted by Bryan Duxbury <br...@rapleaf.com>.
I don't have metrics on hand, but you almost always want to wrap your base
transport with a buffered or framed transport. At least in my experience
with Ruby and Java, a buffer makes all the difference.
-Bryan
On Sun, Mar 7, 2010 at 7:57 PM, <ma...@bestweb.net> wrote:
>
>
>
> Does anyone have any performance metrics for Thrift generated
> client-side code? C++ would be nice.
> As you might guess, I'm trying
> to estimate the difference in the performance of the Thrift generated C++
> code for TBinaryProtocol over TSockets or TBufferedTransports and the C
> code generated by cthrift.
>
> Thanks!
> Mayan
>
>
>
Performance of Thrift C++ client
Posted by ma...@bestweb.net.
Does anyone have any performance metrics for Thrift generated
client-side code? C++ would be nice.
As you might guess, I'm trying
to estimate the difference in the performance of the Thrift generated C++
code for TBinaryProtocol over TSockets or TBufferedTransports and the C
code generated by cthrift.
Thanks!
Mayan
[jira] Updated: (THRIFT-728) Make generated Haskell code an
instance of Arbitrary
Posted by "Aran Donohue (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aran Donohue updated THRIFT-728:
--------------------------------
Attachment: arbitrary.patch
The patch mentioned in the issue body text.
> Make generated Haskell code an instance of Arbitrary
> ----------------------------------------------------
>
> Key: THRIFT-728
> URL: https://issues.apache.org/jira/browse/THRIFT-728
> Project: Thrift
> Issue Type: New Feature
> Components: Compiler (Haskell), Library (Haskell)
> Environment: All
> Reporter: Aran Donohue
> Priority: Minor
> Attachments: arbitrary.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> The patch
> * Generates Arbitrary instances for Thrift structs, enums and exceptions
> * Provides Arbitrary instances for GHC's Int64, Data.Map.Map and Data.Set.Set
> * Makes Thrift enums instances of Bounded and improves the Enum instance declaration
> Making a type an instance of Test.QuickCheck.Arbitrary specifies how to generate random instances of the struct. This is useful for testing.
> For example, consider the following simple Thrift declaration:
> {code:title=distributed_log.thrift}
> enum LogLevel { DEBUG, INFO, WARNING, ERROR }
>
> typedef i32 UnixTimeStamp
>
> struct LogEntry {
> 1: LogLevel level,
> 2: UnixTimeStamp timestamp,
> 3: string message
> }
>
> service Logger {
> void log(1: LogEntry entry)
> }
> {code}
> With the patch, the following program (import statements elided) is a fuzzer for the log service.
> {code:title=LogTest.hs}
> -- ripped from Test.QuickCheck.Gen.sample'
> infexamples ∷ Gen a → IO [a]
> infexamples (MkGen m) =
> do rnd ← newStdGen
> let rnds rnd = rnd1 : rnds rnd2 where (rnd1, rnd2) = split rnd
> return [(m r n) | (r, n) ← rnds rnd `zip` [0,2..] ]
>
> infentries = infexamples (arbitrary ∷ Gen LogEntry)
>
> main = do entries ← infentries
> forM entries logAnEntry
>
> logAnEntry entry = do
> transport ← hOpen ("localhost", PortNumber 9090)
> let binProto = BinaryProtocol transport
> let client = (binProto, binProto)
> (Client.log client) entry
> tClose transport
> `Control.Exception.catch`
> (λ(TransportExn s t) → print s)
> --On systems with Haskell we could just generate a fuzzer like this one.
> {code}
> In implementing the Arbitrary instances, it was useful to make Thrift enums instances of Bounded and to improve the Enum instance. Specifically, whereas before,
> {code}
> [DEBUG .. ]
> {code}
> would throw an exception, now it behaves as expected without an exception.
> I consider the patch incomplete. It's more of a starting point for a discussion at this point than a serious candidate for inclusion. If it is of interest, I'd appreciate some direction on testing it as well as style, and I'd welcome any other comments or thoughts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-728) Make generated Haskell code an
instance of Arbitrary
Posted by "John Billings (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904136#action_12904136 ]
John Billings commented on THRIFT-728:
--------------------------------------
Aran, I think that Thrift.Arbitraries is missing from your patch.
> Make generated Haskell code an instance of Arbitrary
> ----------------------------------------------------
>
> Key: THRIFT-728
> URL: https://issues.apache.org/jira/browse/THRIFT-728
> Project: Thrift
> Issue Type: New Feature
> Components: Haskell - Compiler, Haskell - Library
> Environment: All
> Reporter: Aran Donohue
> Priority: Minor
> Attachments: arbitrary.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> The patch
> * Generates Arbitrary instances for Thrift structs, enums and exceptions
> * Provides Arbitrary instances for GHC's Int64, Data.Map.Map and Data.Set.Set
> * Makes Thrift enums instances of Bounded and improves the Enum instance declaration
> Making a type an instance of Test.QuickCheck.Arbitrary specifies how to generate random instances of the struct. This is useful for testing.
> For example, consider the following simple Thrift declaration:
> {code:title=distributed_log.thrift}
> enum LogLevel { DEBUG, INFO, WARNING, ERROR }
>
> typedef i32 UnixTimeStamp
>
> struct LogEntry {
> 1: LogLevel level,
> 2: UnixTimeStamp timestamp,
> 3: string message
> }
>
> service Logger {
> void log(1: LogEntry entry)
> }
> {code}
> With the patch, the following program (import statements elided) is a fuzzer for the log service.
> {code:title=LogTest.hs}
> -- ripped from Test.QuickCheck.Gen.sample'
> infexamples ∷ Gen a → IO [a]
> infexamples (MkGen m) =
> do rnd ← newStdGen
> let rnds rnd = rnd1 : rnds rnd2 where (rnd1, rnd2) = split rnd
> return [(m r n) | (r, n) ← rnds rnd `zip` [0,2..] ]
>
> infentries = infexamples (arbitrary ∷ Gen LogEntry)
>
> main = do entries ← infentries
> forM entries logAnEntry
>
> logAnEntry entry = do
> transport ← hOpen ("localhost", PortNumber 9090)
> let binProto = BinaryProtocol transport
> let client = (binProto, binProto)
> (Client.log client) entry
> tClose transport
> `Control.Exception.catch`
> (λ(TransportExn s t) → print s)
> --On systems with Haskell we could just generate a fuzzer like this one.
> {code}
> In implementing the Arbitrary instances, it was useful to make Thrift enums instances of Bounded and to improve the Enum instance. Specifically, whereas before,
> {code}
> [DEBUG .. ]
> {code}
> would throw an exception, now it behaves as expected without an exception.
> I consider the patch incomplete. It's more of a starting point for a discussion at this point than a serious candidate for inclusion. If it is of interest, I'd appreciate some direction on testing it as well as style, and I'd welcome any other comments or thoughts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-728) Make generated Haskell code an
instance of Arbitrary
Posted by "Aran Donohue (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aran Donohue updated THRIFT-728:
--------------------------------
Attachment: arbitrary.patch
Include's Iain's feedback and a fix replacing use of the variable "rec" with "record" to allow compilation on GHC 6.12
> Make generated Haskell code an instance of Arbitrary
> ----------------------------------------------------
>
> Key: THRIFT-728
> URL: https://issues.apache.org/jira/browse/THRIFT-728
> Project: Thrift
> Issue Type: New Feature
> Components: Compiler (Haskell), Library (Haskell)
> Environment: All
> Reporter: Aran Donohue
> Priority: Minor
> Attachments: arbitrary.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> The patch
> * Generates Arbitrary instances for Thrift structs, enums and exceptions
> * Provides Arbitrary instances for GHC's Int64, Data.Map.Map and Data.Set.Set
> * Makes Thrift enums instances of Bounded and improves the Enum instance declaration
> Making a type an instance of Test.QuickCheck.Arbitrary specifies how to generate random instances of the struct. This is useful for testing.
> For example, consider the following simple Thrift declaration:
> {code:title=distributed_log.thrift}
> enum LogLevel { DEBUG, INFO, WARNING, ERROR }
>
> typedef i32 UnixTimeStamp
>
> struct LogEntry {
> 1: LogLevel level,
> 2: UnixTimeStamp timestamp,
> 3: string message
> }
>
> service Logger {
> void log(1: LogEntry entry)
> }
> {code}
> With the patch, the following program (import statements elided) is a fuzzer for the log service.
> {code:title=LogTest.hs}
> -- ripped from Test.QuickCheck.Gen.sample'
> infexamples ∷ Gen a → IO [a]
> infexamples (MkGen m) =
> do rnd ← newStdGen
> let rnds rnd = rnd1 : rnds rnd2 where (rnd1, rnd2) = split rnd
> return [(m r n) | (r, n) ← rnds rnd `zip` [0,2..] ]
>
> infentries = infexamples (arbitrary ∷ Gen LogEntry)
>
> main = do entries ← infentries
> forM entries logAnEntry
>
> logAnEntry entry = do
> transport ← hOpen ("localhost", PortNumber 9090)
> let binProto = BinaryProtocol transport
> let client = (binProto, binProto)
> (Client.log client) entry
> tClose transport
> `Control.Exception.catch`
> (λ(TransportExn s t) → print s)
> --On systems with Haskell we could just generate a fuzzer like this one.
> {code}
> In implementing the Arbitrary instances, it was useful to make Thrift enums instances of Bounded and to improve the Enum instance. Specifically, whereas before,
> {code}
> [DEBUG .. ]
> {code}
> would throw an exception, now it behaves as expected without an exception.
> I consider the patch incomplete. It's more of a starting point for a discussion at this point than a serious candidate for inclusion. If it is of interest, I'd appreciate some direction on testing it as well as style, and I'd welcome any other comments or thoughts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.