*** SachaZzz is now known as SachaS
*** johannesV has joined #gnuenterprise
<johannesV> good morning
*** btami has joined #gnuenterprise
<btami> good morning
*** reinhard has joined #gnuenterprise
<reinhard> good morning all
*** johannesV_ has joined #gnuenterprise
*** johannesV has quit IRC
*** kilo has joined #gnuenterprise
<kilo> good morning
*** lekma has joined #gnuenterprise
<lekma> good morning
<lekma> reinhard: i tested you changes, the script that use to take 4min+ is now 0:01:53.797909
<lekma> \o/
<reinhard> excellent
<reinhard> now how do we stand in that sap comparsion?
<reinhard> not that i would take beating sap as a minimum requirement, but just curious
<lekma> First try:
<lekma> 1 min 45 sec. for 516.390 records and display two fields.
<lekma> Second try:
<lekma> 1 min 35 sec
<lekma> so we're not there yet
<reinhard> this is *including* the rpc layer?
<lekma> yep
<reinhard> xmlrpc or hessian?
<lekma> and my test is only 185000 records, one field
<lekma> sap, i don't remember (their protocol), gnue, xmlrpc for the moment
<lekma> i'm hitting strange bugs in http 1.1 with hessian
<reinhard> ok
<reinhard> I think that xmlrpc still adds a lot of overhead
<lekma> even thougt in oure encode/decode hessian is 3 to 4 time faster
<lekma> s/oure/pure
<lekma> the request still eats 300Meg of memory (instead of 500)
<reinhard> well
<reinhard> at least for speed we're not magnitudes away any more
<lekma> :)
*** btami has quit IRC
<reinhard> lekma: as for "only one field", please don't forget that there is always the gnue_id as a second field
<reinhard> also, I think adding more fields wouldn't make it slower proportionally
<reinhard> i.e. a second field would by far not add the same amount of time again
<lekma> no, of course
<lekma> i need a gnue expert
<lekma> that's plain crazy
<reinhard> you mean you need a gnue expert who's plain crazy? ;-)
<reinhard> that would be implied :)
<lekma> a hessianlib.dumps() of a big list outside of gnue context:
<lekma> hessianlib.dumps() 0:00:00.143382
<lekma> the same in ServerAdapter:
<lekma> hessian.dumps () 0:00:26.166066
<lekma> i don't understand where the extra time comes from
<lekma> while xmlrpc is coherent
<lekma> ie:
<lekma> xmlrpclib
<lekma> .dumps:  0:00:02.267522
<lekma> in ServerAdapter:
<lekma> xmlrpc.dumps () 0:00:02.914790
<reinhard> how did you get these times?
<reinhard> pstats module?
<lekma> dumping value :
<lekma>     r = 30000
<lekma>     v = ["this is a record", 1, 2, 1000001, "(08)123123123", "some more text", 22/7.0 , u"some unicode", True, False]
<lekma>     value = ([v for i in range(r)])
<lekma> nope datetime.datetime()
<reinhard> ah
<reinhard> even with pstats i see "unexplainable" results sometimes
<lekma> the crazier is if i run in profile
<reinhard> like a function that only consists of a single function call uses 20 seconds
<reinhard> for 100000 calls
<lekma> everything is back to normal, ie hessian faster than xmlrpc
<reinhard> I suspect all the hidden python magic like garbage collection and the like has a big impact on performance
<reinhard> and the decision when exactly python runs the garbage collection might depend on conditions that we as humans would regard as "random"
*** kilo has left #gnuenterprise
<lekma> it is systematic
<lekma> it is always slower when run in gnue
<lekma> and always faster outside gnue
<lekma> that doesn't seem like random
<lekma> :(
<lekma> it drives me crazy
<reinhard> I think it is slower because within gnue there is a lot more garbage to collect
<reinhard> but I can only suspect, too
<reinhard> that's all guessing
<jcater> could it have anything to do with unicode vs non-unicode data?
<reinhard> AFAICT lekma's non-gnue test contains unicode data, too
<lekma> yes same data
<lekma> everything is equal
<jcater> do you have psyco installed on your computer? I know gnue initializes it by default if it can find it, which in most cases makes a huge performance diff ... but I wonder if it causes problems with your lib
* jcater is just making guesses here ... sorry if I'm butting in :)
<reinhard> jcater: your suggestions are welcome
<lekma> >>> import psyco
<lekma> Traceback (most recent call last):
<lekma>   File "<stdin>", line 1, in ?
<lekma> ImportError: No module named psyco
<lekma> >>>
<lekma> so psyco seems out of the equation
<lekma> could you guys have a look at the code cause it really is driving me crazy
<lekma> ?
<jcater> I'd be happy to
<jcater> as you have me very curious
<reinhard> I can also try
<lekma> can i dcc send?
<reinhard> can you mail?
<jcater> please email to jason at gnue.org for me
<reinhard> reinhard at gnue dot org
<reinhard> :)
<lekma> ok
<lekma> sent
* lekma hopes you will find where he has been stupido :)
<lekma> bbl
<lekma> back
*** johannesV__ has joined #gnuenterprise
*** johannesV_ has quit IRC
<reinhard> lekma: get it
<reinhard> erm
<reinhard> got it
<reinhard> will look this evening (i think)
<lekma> thx
<reinhard> but from a first sight, it looks non-trivial :)
<reinhard> so I really need some time to dig into it
*** derek has joined #gnuenterprise
*** klasstek has joined #gnuenterprise
<lekma> bb
*** johannesV__ has quit IRC
*** bigbrother has joined #gnuenterprise
*** SachaS is now known as SachaZzz
*** klasstek has quit IRC
*** jcater has joined #gnuenterprise
*** bigbrother` has joined #gnuenterprise
*** bigbrother has quit IRC
*** derek has quit IRC
*** sjc has quit IRC
*** derek has joined #gnuenterprise
*** someon has joined #gnuenterprise
<someon> Wow...  been forever since I've been on...  I've been reading the logs, though...
<someon> jamest: Any chance of getting the www->irc proxy back?
<someon> And another question for those that write often....  (Reinhard, Johannes, Lekma, ...) does anyone use Eclipse and PyDev to GNUe?
*** someon has left #gnuenterprise
<jcater_> I tried it
<jcater_> the frustration I had was that a file had to be part of a project
*** jcater_ has quit IRC