*** mnemoc_ has joined #gnuenterprise *** mnemoc_ has quit IRC *** mnemoc has quit IRC *** titopbs has quit IRC *** mnemoc has joined #gnuenterprise *** crazyzivi has joined #gnuenterprise *** crazyzivi has left #gnuenterprise *** reinhard has joined #gnuenterprise *** sjc has joined #gnuenterprise *** btami has joined #gnuenterprise good morning * btami is reading logs tiredbones: http://www.gnuenterprise.org/tools/forms/docs/Developers-Guide.sxw is what you are looking for morning btami good morning hi reinhard lekma: you're already running threaded? i try to but only the xul engine atm i really would like to thread the xmlrpc server but i'm lost in gnue code there's only one thread at a time used by xmlrpc server hi lekma, reinhard which slows down all sessions *** dcmwai has quit IRC morning lekma: making a program like gnue-appserver multithreaded is probably a big and complex task this i did consider already but there are small possible things to do let me look again at your traceback i think at least reinhard: thx making it multithreaded would probably expose hidden bugs :) :) a few of them seem to be hidden (quite well) in DBSIG2 *** kilo has joined #gnuenterprise lekma: are you aware of the fact that calling functions like find() or new() call stuff throughout appserver? that means that if you call find() or new() in different threads, you in fact run appserver multithreaded yep this i know but my prob my main prob is that xmlrpc only answer one request at a time so even if xul engine seems to be threaded yes I see if one user sends a long request to the appserver the rest of the users are blocked for that time you have to wait for every request to finish correct? yep well yes that means that if you call find() or new() in different threads, you in fact run appserver multithreaded that's what i do with xul engine ok it a kind of appserver client that thread every possible requests good morning mornng kilo s/mornng/morning * lekma is tired, again a sleepless night working on appserver :) hi lekma reinhard: what should I do to refresh Forms' DevGuide that is linked on the web? http://www.gnuenterprise.org/tools/forms/docs/Developers-Guide.sxw is older than the current one in SVN upload? ? kilo: you have an account on ash? aye upload the current version to ash /var/www/tools/forms/docs/ lekma: the traceback you get is impossible... should i scp it there? ain't it linked somehow? strange... kilo: I guess it ain't hmmm lekma: self._dataObject gets initialized at object creation for ResultSet so self._dataObject *can* never be None this is absolutely weird either we face a race condition here well here is another one... DB000: Exception in thread Thread-2660: DB000: Traceback (most recent call last): DB000: File "/usr/lib/python2.3/threading.py", line 436, in __bootstrap DB000: self.run() DB000: File "/var/lib/gnue/gecocer/xul/forms.py", line 231, in run DB000: self.obj = self.attr.getXulAttr (self.obj, self.item) DB000: File "/var/lib/gnue/gecocer/xul/attributes.py", line 146, in getXulAttr DB000: attribs = self.__session.find (klass, cond, [], props) DB000: File "/usr/lib/gnue/appserver/language/Session.py", line 119, in find DB000: properties) DB000: File "/usr/lib/gnue/appserver/language/ObjectList.py", line 48, in __init__ DB000: self.__buildList () DB000: File "/usr/lib/gnue/appserver/language/ObjectList.py", line 108, in __buildList DB000: self.__populateList () DB000: File "/usr/lib/gnue/appserver/language/ObjectList.py", line 118, in __populateList DB000: rset = sm.fetch (sid, self.__list_id, len (self.__list), self.__cacheStep,0) DB000: File "/usr/lib/gnue/appserver/geasSessionManager.py", line 300, in fetch DB000: result = s.fetch (list_id, start, count) DB000: File "/usr/lib/gnue/appserver/geasSession.py", line 467, in fetch DB000: result = list.fetch (start, count) DB000: File "/usr/lib/gnue/appserver/geasList.py", line 260, in fetch DB000: self.__fillupFunc (start + count) DB000: File "/usr/lib/gnue/appserver/geasList.py", line 76, in __fillupFunc DB000: return self.__fillupUnsorted (count) DB000: File "/usr/lib/gnue/appserver/geasList.py", line 133, in __fillupUnsorted DB000: current = self.__getInstance () DB000: File "/usr/lib/gnue/appserver/geasList.py", line 163, in __getInstance DB000: record = self.__recordset.nextRecord () DB000: File "/usr/lib/gnue/appserver/data.py", line 973, in nextRecord DB000: rec = self.__nextBackendRecord () DB000: File "/usr/lib/gnue/appserver/data.py", line 1087, in __nextBackendRecord DB000: beRec = self.__resultSet.nextRecord () DB000: AttributeError: 'NoneType' object has no attribute 'nex DB000: rec = self.__nextBackendRecord () DB000: File "/usr/lib/gnue/appserver/data.py", line 1087, in __nextBackendRecord DB000: beRec = self.__resultSet.nextRecord () DB000: AttributeError: 'NoneType' object has no attribute 'nextRecord' pasted in two times too long hrmmm if I only knew more about multithreading :) different threads run in the same address space, right? unfortunately they share all memory i don't know more than you i think so they run in main thread memory space i think you know a few month ago you would have ask me what was the meaning of programming i would have answered uh?? :) lekma: you joking? nope i learned by doing which tells a lot about the quality of my code :) *lol* brb, smoke break :) I don't think there's a connection between that I program since I was 12 and my code isn't that good, either :) better than mine ;) reinhard: you started with what kind of machine? commodore vic 20 then commordore 64, atari 800xl, dec vax at school, then pc hehe, i started on RXX (dont remember exact number) via terminals, then ZX81, Spectrum, C64, then pc reinhard: there should be an easy way to thread xmlrpc server but i'm really lost in all those common.rpc files i tried to do it with SimpleXMLRPCServer.py and it works (only by replacing to occurences of SocketServer.TCPServer by SocketServer.ThreadingTCPServer) but with gnue common i feel in the middle of a labyrinth too much layer of abstraction gives me a headache :) after the fifth file open chasing server init i was wondering where i started already :) well known gnue feature... 8-)) lekma: now you see what I have to face every day I have to *code* against common... lol heh reinhard: you love it ;) seriously I don't rewrite common? :) * reinhard would like to point out that most of the code in common was not done by him... rewrite common in C? appserver has already been tried in C then in assembly 8-)) seriously i dont know if it could earn something by that... so it is only joke rewrite it in java & be done with it brrrrrrrr lekma: I think one of the problem fields with multithreading is say you request a list and in another thread you fetch data from that list the request() may not be through but actually I think what I say makes no sense because you can't fetch() without waiting the request() to return and count() i think I think you will get problems as soon as you act on the same session from different threads well honestly I think you will always have problems as soon as you use threads at all :( has anybody here at all ever written a truely multithreaded program? well at least we could have one thread per session instead of one for all of them atm i only use threads for session.find() and they never try to get the same data even though they access the same classes and these find()s run on the same session or on different sessions? same session well it's always one session at a time atm so they can't get in the way of each other in fact that's what i'm trying to find out how i can get them to step on each other toes :) maybe i'll get a nice firework has anybody here at all ever written a truely multithreaded program? i did, in 1995, in C but it was only about 100 lines... lekma: currently, it looks very much to me as if two different threads would fetch from the same list first of all, you could check if really all lists get an unique list id :q err wrong window which debug level should i use?? 456 or 4,5,6 or 4-6 you get the list id as result of request() in your own code i use find() only atm for threads it doesn't return list id is it?? ah no for one request from xmlrpc client there dozens of find so list id jumps from 873 to 956 all missing are find i think lekma: in language/ObjectList.py you can see where the find in fact alls a request() and the list id is fed into self.__list_id you can add a print self.__list_id there for example reinhard: well they seem to have unique id that's good the error occur between 450 and 451 you can reproduce the error? same error every time? yep it's not happening always but when it happens it's always at the same request what are the new debug-level to see just sql request?? 3 list ids should be unique throughout one session, correct? yes *** sjc has quit IRC lekma: next random guess: do you close sessions or lists manually in your application? yep exactly your effect should be reproducible by closing a session while another thread still runs a find() in that session i don't close session but lists ok the same holds for closing lists while there's still some processing running in that list + def closeLists (self): + + for item in self.__lists.values (): + item.close () in geasSession oh that looks very dangerous closing the list frees quite some objects in memory that's the point that are needed in find() and such otherwise there's memory use inflation :) i'll try without the closeList but it seems odd to me cause i wait for all threads to finish before returning anything and then when there is result i closeLists well, what I can say quite surely is that geasList.close() runs too early ok i'll test it thx for helping either that or the last part of __getInstance those are the only 2 points where geasList.__recordset is set to None and one of your exceptions IIRC clearly stated that geasList.__recordset is None the last part of __getInstance <-- ?? in geasList?? yes that part where self.__recordset is set to None actually that part should only run when all record have been fetched anyway and you are at the end of the list so I much more suspect close() than that nope without closeLists triggered, i still have the same error lekma: you could add a test print at both points to find out which of them actually gets executed maybe with printing the list id bad thing is I'm not sure whether multithreaded programs show their output in the true order not really reinhard: do you have any idea what the following could mean in less rich words File "/usr/lib/gnue/appserver/language/Session.py", line 119, in find properties) File "/usr/lib/gnue/appserver/language/ObjectList.py", line 48, in __init__ self.__buildList () File "/usr/lib/gnue/appserver/language/ObjectList.py", line 106, in __buildList self.sortOrder, self.properties) File "/usr/lib/gnue/appserver/geasSessionManager.py", line 272, in request result = s.request (classname, conditions, sortorder, propertylist) File "/usr/lib/gnue/appserver/geasSession.py", line 391, in request recordset = self.__connection.query (content, dsCond, sortlist) File "/usr/lib/gnue/appserver/data.py", line 645, in query content, _conditionTree (conditions), order) File "/usr/lib/gnue/appserver/data.py", line 939, in __init__ self.__mergeWithCache (conditions, order) File "/usr/lib/gnue/appserver/data.py", line 1153, in __mergeWithCache self.__removeFilter (row, conditions) File "/usr/lib/gnue/appserver/data.py", line 1205, in __removeFilter if not condition.evaluate (lookup): File "/usr/lib/gnue/common/datasources/GConditions.py", line 120, in evaluate if not child.evaluate (lookup): File "/usr/lib/gnue/common/datasources/GConditions.py", line 120, in evaluate if not child.evaluate (lookup): File "/usr/lib/gnue/common/datasources/GConditions.py", line 577, in evaluate GBinaryConditionElement.evaluate (self, lookup) File "/usr/lib/gnue/common/datasources/GConditions.py", line 380, in evaluate self.values = unify ([child.evaluate (lookup) for child in self._children]) File "/usr/lib/gnue/common/datasources/GConditions.py", line 1147, in unify __unify (values, result) File "/usr/lib/gnue/common/datasources/GConditions.py", line 1245, in __unify raise ConversionRuleError, (oldValue, chkValue) ConversionRuleError: No unification rule for combination 'int' and 'bool' that means that you try to compare an integer with a boolean in a condition in a find() condition?? http://uml2svg.sourceforge.net/ <--- gsd2xmi? lekma: yes, in a find() condition mnemoc: hmmm, maybe i don't get it, i don't compare oranges and apples and still i get this mysterious error what *do* you compare? strings with strings, bool with bool, ... can you find out exactly which condition causes this error? yep i'll try right now bbl *** kilo has quit IRC SELECT t1.docs_docId,t2.gnue_id,t1.gnue_id,t0.gnue_id FROM docs_Series t0 LEFT JOIN docs_Docs t1 ON t1.gnue_id = t0.docs_docId LEFT JOIN docs_Docs t2 ON t2.gnue_id = t0.docs_serieId WHERE (((t2.docs_docId = '600003'))) that's the last select before error there are not even bool or int and in db it works brb phone I was using gnue-designer and took a segfault. I was doing my third unbound entry. Error :TODO UPDATING VALID SET lekma: you should be able to find out which find() this is the error happens in an appserver-evaluated part of the condition let me explain when you give a condition to find, it gets split into a part that the db must handle, and another part that appserver must handle (e.g. because it contains calculated fields) what you see in the select is the db part of the condition appserver goes through every record of the result set and checks if it's part of the condition is ok or not and discards the records where the "appserver-part" of the condition is false well i found out which part of condition was a pb but it's still odd i was comparing a boolean to True it's a boolean in db it's a boolean in gcd odd it's a boolean in gnue_property *** btami has quit IRC i can't compare anymore to True or False in bound proc i always got this error that sucks i'm tired :( let me check something lekma: I think I found it lekma: can you please svn up common/rcp and try again? ok and if it doesn't work, try rpc instead of rcp ;-) how can i selectively svn only a folder ?? go to that folder and "svn up" there nope doesn't work hrmm... must try to reproduce just to make sure: "svn up" doesn't work or condition doesn't work? session.find () in onvalidate with a cond like {'class_prop': True} where module_class is bool condition doesn't work ok is there a debug-level available to see all the non xmlrpc stuff appserver does?? brb reinhard: can you reproduce or am slowly beginning to go down the path of insanity?? :) *** SachaS has joined #gnuenterprise *** titopbs has joined #gnuenterprise lekma: I can not reproduce sorry for my bad response time (even worse than appserver) - I'm doing 3 or 4 things at the same time ok, that's official, i am insane :) no a few question the boolean field you check is a db field or a calculated field? db does the class contain dirty instances at the time you read it? or is any calculated field involved in the same condition? no calculated field, but at the time of validation self is in dirty instance which on second read seems obvious lekma: are you sure this worked before? and if yes can you say when it last worked? yes i am i didn't need it for a long time (at least few weeks) ok so it could be that it broke with dirty reads? so i don't remember when it last worked maybe myArt = session.find ('articles_Articles', {'articles_docId.docs_docId': serie.docs_docId.docs_docId, \ 'articles_version.articles_varKeyId': self.articles_version.articles_varKeyId, \ 'articles_support.articles_varKeyId': self.articles_support.articles_varKeyId, \ 'articles_standard.articles_varKeyId': self.articles_standard.articles_varKeyId}, [], []) this works I think I found a solution just a second but if i add 'articles_valid': True it borks can you please svn up in gnue-common/src/datasources and try again? ok __unify (values, result) File "/usr/lib/gnue/common/datasources/GConditions.py", line 1273, in __unify __unify (values, result) File "/usr/lib/gnue/common/datasources/GConditions.py", line 1158, in __unify checktype (values, types.ListType) RuntimeError: maximum recursion depth exceeded i just pasted the last lines weee *** jcater has joined #gnuenterprise :) *** jcater has quit IRC *** jcater has joined #gnuenterprise *** ChanServ sets mode: +o jcater *** jcater_ was kicked by jcater (jcater) *** jcater sets mode: -o jcater *** dcmwai has joined #gnuenterprise lekma: next try can you again svn up in datasources? gnue-common/src/datasources?? (sorry for this tral and error but I can't easily reproduce) yes np, i'm so glad you help no luck doesn't work i think i will let it sink for today and check it back tomorrow what symptom? File "/usr/lib/gnue/common/datasources/GConditions.py", line 1264, in __unify raise ConversionRuleError, (oldValue, chkValue) ConversionRuleError: No unification rule for combination 'bool' and 'int' i really don't get it maybe i'm too tired i don't see the obviou s sorry spotted a typo lekma: you have power for another try? ok fix committed .. .... now reinhard: you're a damn genius!!! it works again lekma: no lekma: i'm a damn fool - would have worked the first time already if i'd fixed it corectly :) :) :) lekma: if I would not have seen in real I would think reinhard and johannes are some form of artificial intelligence machines ;) virtual ones what still strikes me odd at some times is that I can look at johannes' code I've never looked at before and withing a few minutes understand it enough to fix something you both make a hell of a team reinhard: i found your quote from Tannenbaum: "The nice thing about standards is that you have so many to choose from. Furthermore, if you do not like any of them, you can just wait for next years's model." SachaS: lol - never heard the second sentence well, actually gnue's line is more like "standards are so cool - everybdoy should have his own" ;-) :) *** wendall911 has joined #gnuenterprise off for toda today cu all *** reinhard has quit IRC so who's the one i talk to about the gnue xmlrpc server?? :) SachaS: are you working on pyro?? i am not working on anything for gnue :( i just hang out here ;) :) *** jamest has joined #gnuenterprise *** nickr_ has joined #gnuenterprise *** nickr has quit IRC *** holycow has joined #gnuenterprise *** sjc has joined #gnuenterprise *** dcmwai has quit IRC *** dimas has quit IRC *** jamest has left #gnuenterprise *** cilkay has quit IRC *** SachaS has quit IRC *** titopbs_ has joined #gnuenterprise *** titopbs has quit IRC *** siesel has joined #gnuenterprise hi lekma, you've interest in rpc? i have questions and pbs :) i wonder if the xmlrpc server is a threaded one cause we're implementing appserver right now at least one of the two appserver versions is threaded. oops, I mean xmlrpc server versions but right now with 10 people working together we never more than one thread used by the server and we all have to wait for the previous call to finish ok, got it. that's pretty annoying reading the code (for what i can read) the server is threaded but that's not what it looks like in prod there's never more than one server thread running could it be that our rpc call are sync calls?? The client side does sync calls, but the server should do it. Seems like I have to test it. i'd be glad if you can help on this one np hey guys a OT question undergoing a mail bomb and wondering if there is a quick way I can disable a domain in postfix, so it drops all mail from it instantly. not a postfix user here * wendall911 hasn't had to do this task before siesel: i'm using xmlrplib from python std if this can help :) ok. I thought so. Do you know how to make python wait? time.sleep cool import time brb *** titopbs_ has quit IRC *** titopbs has joined #gnuenterprise lekma, it works, the xmlrpc server is threaded wow can i svn up ? *** kilo has joined #gnuenterprise sorry,i didn't use appserver, but an test programm i'm eager to test it :) brb *** jcater has quit IRC lekma: I've tried it with appserver, and it seems to work too. great :) I've used the following trick add a line "time.sleep" to one of appserver functions (in geasSession.py) ?? how much time.sleep?? :) then call "testRPC.py" on two different terminals. Then you can see, that the call (with time.sleep) from the first one will be interupted by the second one 6 oh you mean without modification the xmlrpc server is threaded yep i was at lost here i thought you made some modif hmmm no, not needed so what we see is kinda odd then cause we send a lot of client rpc sync calls and the server never runs more than one thread to answer all these calls do you have a hint ? how do you know thats allways just one thread? watching top there's the 3 main threads always running but for each call there's one thread showing up then finishing then next one never more than one at a time i'm running a threaded client to appaserver and when it lauches 10 or more threads you can feel the difference feel and see actually how do you see the number of threads in top? I never cared for *** holycow has quit IRC *** titopbs has quit IRC n8t *** siesel has quit IRC *** lekma has quit IRC *** wendall911 has left #gnuenterprise *** ajmitch has quit IRC *** jamest has joined #gnuenterprise *** kilo has quit IRC *** sjc has quit IRC *** dcmwai has joined #gnuenterprise *** ajmitch has joined #gnuenterprise *** dcmwai has quit IRC *** titopbs has joined #gnuenterprise *** dcmwai has joined #gnuenterprise *** titopbs has quit IRC *** dcmwai has quit IRC *** jcater has joined #gnuenterprise *** holycow has joined #gnuenterprise *** jcater has quit IRC *** dcmwai has joined #gnuenterprise *** tiredbones has quit IRC