Magnus Lyckå
(Started late due to projector problems)
[eichin:20040609T0923+02]
Axes: standalone database server vs. embedded in process
There are still a lot of quirks in the python db apis (we knew this.)
import mx.ODBC.Windows as dbi is a reasonable use of aliasing.
No embedding, explicit-strings in an execute() method; no
consistent substitution-syntax.
dbi-style interfaces don't default to autocommit.
There are standardized exceptions.
[eichin:20040609T0934+02]
Dr. Duncan Grisby duncan@tideway.com
Tideway Systems: software to help large orgs undestand and manage
their IT application infrastructure; venture funded, hiring in
London.
Many node types ("apache", "workstation", "Fred"), complex data in
each node, highly interconnected, cross-platform access to data; also
needs to support distributed transactions.
SQL sucks for graph traversal - complex to write, slow to execute
(there are higher level tools to build it, but the results are hard to
maintain.)
Berkeley DB - needs commercial license for what they're doing (which
they're ok with.)
Can store multiple tables in a file; can use it in a mode where
multiple stores lead to multiple values per key; can do "find keys
beginning with this substring"... also supports some kind of table
cross-linking. Alternative storage modes (btree, for example) are
very useful for performance (locality) tuning.
Each "node" is a CORBA object - gives universal access, can pass
object references around directly. Each node state is also a set of
key-value (string/CORBA-ANY) pairs which also lets other languages
look at it. They can be manipulated in sets; full text search, and
other APIs, are implemented on top.
Node is pretty obvious, but also has get_edges. Their
edge-storage model is proprietary, though.
(My question:) Locking is done at the Berkeley DB layer, though they
also have a higher-level transaction layer.
CORBA "default servant" python object representing all nodes.
(Effectively a cursor?)
Keys are id + "kind" (single char tag for the kind of value); values
are CORBA self-describing syntax. For example:
NNNt Host
NNNk Hostname,ip,os
NNN_hostname fred
etc. Since they're near each other lexicographically, they have
locality in caching too, because of the use of btree (ie. intentionally.)
Tim Bray's http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC page
was useful for figuring out how to index.
BerkDB transactions - 2PC, pessimistic locking, may raise deadlock
exception so you have to have retry handlers.
DB cache size is important - default is 256K, 512M works better :-)
Conclusions: relational isn't the right solution for some things; Berk
DB and CORBA are good building blocks; Python is a great language for
this kind of thing.
Tommi Virtanen tv@debian.org
(Had trouble with the mac - drove the display but only gave an aqua
screen, possible resolution problem... ended up ethernet-sharing with
an RH9 laptop.)
17K lines of python, including docs and examples - probably the
largest Twisted app ever.
Has simple interface for Distinguished Name objects.
Pure python - "no segfaults, unlike OpenLDAP" :-) Also stylistically
pythonic, not wrapped-C-API.
Stringized result objects are LDIF.
Uses Nevow to build a web app to do LDAP lookups from a form (see
slides on the Wiki)
See LDAP lightning talk later "grok LDAP fast".
"If you're doing networking with python at all, You must learn
Twisted. It took about 5 hours to learn the essentials."
Lennart Ohlsson - Lund University
Interface for exploiting the kind of things modern graphics cards
(Programmable GPUs in particular) can do, shading and reflectivity
tricks, volumetric 3d.
Shaders and such are simple, applications integration is hard and
messy.
Existing frameworks (CgFX, DirectX Effects) built around the "Effect"
abstraction, but still an insufficient mini-language.
PyFX - use python to actually specify the effects, instead of an adhoc
minilanguage. Framework as well. Currently on PyOpenGL and SWIG of
Cg for the runtime. Simple effects look a lot like CgFX.
Generic Application Interface - a simple rendering loop.
GPGPU - using the GPU as a number cruncher. (Hypnotic swirl effects,
but it isn't clear how much it really needs the GPU)
http://graphics.cs.lth.se/pyfx/
(see Wiki for references, later)
[eichin:20040609T1146+02]
(previous talk ran late)
Wayne Boucher, Cambridge
http://www.ccpn.ac.uk/
CCPN: Collaborate Computing Project for NMR
MEMOPS: MEtaMOdelling Programming System
10% of the audience of X-ray crystallography, so there are fewer tools
and more of them are adhoc and proprietary.
Standardized on a data model rather than format because it
describes the domain, and is more accessible to other languages and
tools. "Model Driven Architecture", using UML and hardcore tools
(Object Domain, in particular, because it uses Python for scripting,
after starting with Rational Rose - which uses VB - "I lasted about
three hours before giving up".)
The UML screenshot really doesn't look very pictorial...
UML usage further constrainted by XML ("parent/child") and SQL (by
having keys - though apparently their SQL users said they wanted
integers instead...)
Initial scripts are directly pulled down from UML, but most people
work directly with the model. (Couldn't use XMI, because it doesn't
do placement, just structure.)
Over 300 classes, 2300 metaobjects
Currently around 370k lines of python (600k in java!) in 22 packages.
They do UML models of new applications too, sometimes. Primary
existing applications are conversion scripts.
Licensing for the generated code is LGPL, scripts will probably be GPL.
[eichin:20040609T1210+02]
Theo de Ridder
SE-RTES - Software Engineering for RealTime Embedded Systems
His experience: Lisp in the 60's, Simula in the 70's, etc.
Hates UML - because it just shows attributes, which are "silly
things", and you don't see Responsibilities or Performance. "If a
child of 6 years gave you a drawing like that, you'd think he had a
handicap..."
Tensions/axes: generic/specific, textual/graphical, physical/virtual
AIBO - 576mhz MIPS R7000, 64M, 802.11b, etc. "Very rich" for
programming.
aibo.cs.uu.nl - Dutch Aibo robocup league
XABSL - XML Agent Behavior Specification Language, used by the German
Team to win in 2003: 250kloc, lots of tooling, modular architecture.
(huge system.) agents, options, symbols, basic behaviors.
Basic behaviours in C++, strategic behaviour declared in XML, XSLT
generated IC from that. 16kloc of XML, low signal/noise ratio, clumsy
and error prone - so, reconsider using python, since it's language
after all.
Tagged Hierarchy syntax. Map ":" to "__", xi__include(href=...)
for example. Since you pass in an object for the "tree" base, you can
do arbitrary things with it on output...
Basically, hairy support structure but clear python resulting
operation code. Complete recursive state machine implemented with
changing-base-classes?
X3D syntax examples - overloaded ~ and using it as a sort of macro
lead-in marker.
He wants a very minimal RTOS that "only runs python", along with a
minimal interpreter for embedded use.
Also wants a 3d blender/world editor that is live, no attribute
editors, so you can interact on either side (python code or images.)
questions: Alice.org (? university 3d programming-teaching project in python?)
Anna to-be-Martelli track lead. 5 minutes to talk, she even has a hooked pole...
Stefan Schwarzer - ftputil: a high level FTP client library
Side track: Pet Peeves survey:
- Spelling
- Projectors don't work
- Talking to the Wall
- Reading Slides
- Mumbling
- Small text
- Speaking too quietly
- Broken Python Code
- Typing code in really slowly in front of the audience
- Deleting source code before running the demo
- Launching *.py with JAVA
- Showing URLs for 2 seconds
Tommi Virtanen tv@havoc.fi: Eunuchs - Missing manly parts of UNIX API for Python
- C/Pyrex bits, cover stdlib missing things
- have them now, not wait for new release: tuntab, fchdir, sendmsg
- low level, dirty, absolutely necessary, "real systems programming"
- Guido: "There's a 2 year old PEP on how to add new libraries, I
don't know what's up with it either"
- send me other small ones as patches.
- Not knowing where the lights are
- Poor timekeeping
- Contrast
- Transitions
- Repetition
- No audience participation
- Excess audience participation
- Avoiding questions
- Not Knowing subject
- Door slamming
- Relying on connectivity
? Astrazenica(?) (one of the sponsors): CLAB
- pharmaceutical company, 60k employees, R&D is 11k, Global R&D IS is 600...
- science teams have university culture which is python friendly
- CLAB is 20k lines of python, 200 unit tests, 175 acceptance tests
- (general success story)
24. Track Chairs who don't know how to work the equipment
24. Missing Track Chairs (moshe?)
Tommi Virtanen tv@havoc.fi: Grok LDAP fast
- using DNS domain as a root is "modern"
- LDIF - lines broken as long as they're indented ("stupid idea" - GvR)
Martijn Faassen, Infrae: lxml - a sane Python wrapper for libxml
- libxml is fast and good, xpath, relaxng, etc.
- current python bindings not pythonic - c-ish, utf-8 not
python-unicode, manual memory management, only documented in C
- currently only has refcounting and ElementTree API
- moving into svn on codespeak.net. All in pyrex
Holger Krekel: SHPY
- pygame-based cell-based editor, gives you a python-interactive "pad"...
- multiple cursors for multiple clients... "latency-free"?
- sending code over the wire.
- "SHARE" server, "execution" server (roughly, python -c "exec input()"
- rough hack done in 4 days by Holger Krekel and Armin Rigo
- can use ssh, other authentication
- in codespeak svn -- but talk to them, it's rough.
- not quite a subethaedit replacement, because it uses a server, but...
- finally, the got it working, and scribble a lot...
Steve Alexander: yagni, whui & don
- You aren't gonna need it - get moving faster
- We haven't used it - means you should have yagni'd more
- Don: Knuth's quote of Hoare about "premature optimisation is the
root of all evil" -- or, Don't Optimize Now
- question: "How to tactfully handle your open source community"
(doesn't have an acronym yet)
Benjamin ?: Pyano
- Python Archive NetwOrk
- CPAN-like for Python
- what about PyPI? It doesn't do dependencies (audience comment, I
think from GvR, was to "work with PyPI instead")
(Jacob distributes prizes from feedback forms - bottle of wine, and
Apress-donated "Practical Python")
Aroldo Souza-Leite: Sonnets from Pythia
- a collection of poems, "by, around and about" the Python Community.
- experiment in (post) modern literature
- not compilable (that's perl)
- not about bad jokes in bad poems
- sonnets
- recited Steve Alexander's painful "A common gateway" sonnet
Michael Hudson: "The Bub's Brothers"
Michael Salib: Insecticide: the ultimate Python debugger
- interviewed with Green Hills, looked at embedded chip trace port idea
- time travel support - 1G on traceboard == 1 minute of execution
- so, just an idea: so add a traceport to the python VM...
- or integrate cleverness in the VM directly
- record complete state
- show me all state-snapshots when a condition is true
- perform execution-diffs!
- object-level history
Michel Marquez:
- actually works on Amaretto Venture project (ooooh)
- massive overhead to write normal programs, even short ones (.pkg, .SIS)
- python packaging could be much simpler
- Hello World is 46 lines of code - but with OK/Cancel buttons, and
a wizard, about 200 lines of code...
- Python 2.2.2, last stable when project started
- 60% of GUI wrapped
- "Not released until Friday" - customer edition available then?
- "under 800k" target - which was easy.
- currently needs the SDK, for the emulator, but it should just
work on the phone
-- break --
Talked to Marquez; talked to Michael Salib
Side track: Good things
- Repeating the question
- Not using the projector
- Speaking to the audience
- Involving the audience
- Using pygame for demos -- "making it fun"
- Maintaining eye contact with audience
- Smiling
- Talking about interesting stuff
- Staying on topic
- AV assistance
greg? - Highlighting python in slides
- tokenize in cookbook
- idle
- SilverCity - built on top of Scintilla
- doesn't need valid code
- has a cgi script, but doesn't need to be
Jim Fulton: ZPackage (Zope Packaging System)
- Zope is an application, not a library, different needs from
distutils
- Zope applications themselves are more like modules, but expect to
be installed into a Zope software area, not site-packages
- Want to reuse distutils
- Wants to be good, like CPAN or apt-get or yum - dependency
modeling, install and download, update
- needed for Zope 3 release
- setup.py written by packaging tool
- this uses packman/pimp
- Modulating your voice
- Making slides readable
- Using graphics on slides, not text
- Showing live demos
- Making people laugh (Harald was giving out chocolate?)
- Explain the target audience
- Give context, clear structure
- Putting stuff in the Wiki
bad side:
- Using proportional fonts to show python interactive sessions
Michael Sparks (BBC R&D): Kamaelia's Component System
- streaming server infrastructure replacement
- open standards: quicktime/real/windows are all proprietary
- scale to millions of concurrent viewers, not thousands (entire UK population)
- Concurrency is the big hit: process/threads too expensive, event
based systems are "obfuscating goto" systems :-)
- but real-world things work concurrently, so it can't be that hard
- Generators with IPC - modeled after pipelines, but multi-dimensional
- things don't know where their inputs/outputs are, just that they
have dataflow from/to them.
- postman, "coordination and tracking"
- service is (component, inbox) tuple
- basic DCT transform component done
- trivial inherit-from-Component and add yield mechanism
- Not released yet, working on open source
- makes maintenance simpler, because it is much more readable
- try/except to warn about threads dying, but keep it running
- started a couple of years ago, stackless not really ready then.
- plan to use it on the front end for streaming, it is the most
expensive part of the system. (questioner mentions a Norwegian
equivalent)
Amir Bakhtiar: open source work at Divmod (divmod.org)
- atop, lupy, etc.
- demo of reverend, lupy, quotient
- [later correction: he's at divmod, which employs key members of
the Twisted team, but he's not himself on it]
- quotient does spam filtering, integrates pictures into galleries,
fully searchable, auto-files lists, does an address book...
- $9.95/mo for the service, or just install it
- and it's his (and the team's) primary email server...
[eichin:20040609T1621+02]
Finally - multiplayer run of Bub's Brothers!
"Something's wrong with your keyboard -- Yeah, it's US layout -- OK,
find me the colon key, then"
"I'm missing out some of the details..." - meaning "leaving out", UK British...
"beamer" - slang for video projector.
"social skills track"
Database Programming with Python
Magnus Lyckå
(Started late due to projector problems) [eichin:20040609T0923+02]
Axes: standalone database server vs. embedded in process
There are still a lot of quirks in the python db apis (we knew this.)
import mx.ODBC.Windows as dbi is a reasonable use of aliasing.
No embedding, explicit-strings in an execute() method; no consistent substitution-syntax.
dbi-style interfaces don't default to autocommit.
There are standardized exceptions.
A Graph-Based Data Store
[eichin:20040609T0934+02] Dr. Duncan Grisby duncan@tideway.com
Tideway Systems: software to help large orgs undestand and manage their IT application infrastructure; venture funded, hiring in London.
Many node types ("apache", "workstation", "Fred"), complex data in each node, highly interconnected, cross-platform access to data; also needs to support distributed transactions.
SQL sucks for graph traversal - complex to write, slow to execute (there are higher level tools to build it, but the results are hard to maintain.)
Berkeley DB - needs commercial license for what they're doing (which they're ok with.)
Can store multiple tables in a file; can use it in a mode where multiple stores lead to multiple values per key; can do "find keys beginning with this substring"... also supports some kind of table cross-linking. Alternative storage modes (btree, for example) are very useful for performance (locality) tuning.
Each "node" is a CORBA object - gives universal access, can pass object references around directly. Each node state is also a set of key-value (string/CORBA-ANY) pairs which also lets other languages look at it. They can be manipulated in sets; full text search, and other APIs, are implemented on top.
Node is pretty obvious, but also has get_edges. Their edge-storage model is proprietary, though.
(My question:) Locking is done at the Berkeley DB layer, though they also have a higher-level transaction layer.
CORBA "default servant" python object representing all nodes. (Effectively a cursor?)
Keys are id + "kind" (single char tag for the kind of value); values are CORBA self-describing syntax. For example:
etc. Since they're near each other lexicographically, they have locality in caching too, because of the use of btree (ie. intentionally.)
Tim Bray's http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC page was useful for figuring out how to index.
BerkDB transactions - 2PC, pessimistic locking, may raise deadlock exception so you have to have retry handlers.
DB cache size is important - default is 256K, 512M works better :-)
Conclusions: relational isn't the right solution for some things; Berk DB and CORBA are good building blocks; Python is a great language for this kind of thing.
Ldaptor - pure python LDAP servers and clients
Tommi Virtanen tv@debian.org
(Had trouble with the mac - drove the display but only gave an aqua screen, possible resolution problem... ended up ethernet-sharing with an RH9 laptop.)
17K lines of python, including docs and examples - probably the largest Twisted app ever.
Has simple interface for Distinguished Name objects.
Pure python - "no segfaults, unlike OpenLDAP" :-) Also stylistically pythonic, not wrapped-C-API.
Stringized result objects are LDIF.
Uses Nevow to build a web app to do LDAP lookups from a form (see slides on the Wiki)
See LDAP lightning talk later "grok LDAP fast".
"If you're doing networking with python at all, You must learn Twisted. It took about 5 hours to learn the essentials."
PyFx
Lennart Ohlsson - Lund University
Interface for exploiting the kind of things modern graphics cards (Programmable GPUs in particular) can do, shading and reflectivity tricks, volumetric 3d.
Shaders and such are simple, applications integration is hard and messy.
Existing frameworks (CgFX, DirectX Effects) built around the "Effect" abstraction, but still an insufficient mini-language.
PyFX - use python to actually specify the effects, instead of an adhoc minilanguage. Framework as well. Currently on PyOpenGL and SWIG of Cg for the runtime. Simple effects look a lot like CgFX.
Generic Application Interface - a simple rendering loop.
GPGPU - using the GPU as a number cruncher. (Hypnotic swirl effects, but it isn't clear how much it really needs the GPU)
http://graphics.cs.lth.se/pyfx/
(see Wiki for references, later)
The MEMOPS Programming Framework
[eichin:20040609T1146+02] (previous talk ran late)
Wayne Boucher, Cambridge http://www.ccpn.ac.uk/
CCPN: Collaborate Computing Project for NMR MEMOPS: MEtaMOdelling Programming System
10% of the audience of X-ray crystallography, so there are fewer tools and more of them are adhoc and proprietary.
Standardized on a data model rather than format because it describes the domain, and is more accessible to other languages and tools. "Model Driven Architecture", using UML and hardcore tools (Object Domain, in particular, because it uses Python for scripting, after starting with Rational Rose - which uses VB - "I lasted about three hours before giving up".)
The UML screenshot really doesn't look very pictorial...
UML usage further constrainted by XML ("parent/child") and SQL (by having keys - though apparently their SQL users said they wanted integers instead...)
Initial scripts are directly pulled down from UML, but most people work directly with the model. (Couldn't use XMI, because it doesn't do placement, just structure.)
Over 300 classes, 2300 metaobjects
Currently around 370k lines of python (600k in java!) in 22 packages.
They do UML models of new applications too, sometimes. Primary existing applications are conversion scripts.
Licensing for the generated code is LGPL, scripts will probably be GPL.
Pythonic Alternatives for declarative XML-based specifications of robot behaviour
[eichin:20040609T1210+02] Theo de Ridder
SE-RTES - Software Engineering for RealTime Embedded Systems
His experience: Lisp in the 60's, Simula in the 70's, etc.
Hates UML - because it just shows attributes, which are "silly things", and you don't see Responsibilities or Performance. "If a child of 6 years gave you a drawing like that, you'd think he had a handicap..."
Tensions/axes: generic/specific, textual/graphical, physical/virtual
AIBO - 576mhz MIPS R7000, 64M, 802.11b, etc. "Very rich" for programming.
aibo.cs.uu.nl - Dutch Aibo robocup league
XABSL - XML Agent Behavior Specification Language, used by the German Team to win in 2003: 250kloc, lots of tooling, modular architecture. (huge system.) agents, options, symbols, basic behaviors.
Basic behaviours in C++, strategic behaviour declared in XML, XSLT generated IC from that. 16kloc of XML, low signal/noise ratio, clumsy and error prone - so, reconsider using python, since it's language after all.
Tagged Hierarchy syntax. Map ":" to "__", xi__include(href=...) for example. Since you pass in an object for the "tree" base, you can do arbitrary things with it on output...
Basically, hairy support structure but clear python resulting operation code. Complete recursive state machine implemented with changing-base-classes?
X3D syntax examples - overloaded ~ and using it as a sort of macro lead-in marker.
He wants a very minimal RTOS that "only runs python", along with a minimal interpreter for embedded use.
Also wants a 3d blender/world editor that is live, no attribute editors, so you can interact on either side (python code or images.)
questions: Alice.org (? university 3d programming-teaching project in python?)