google SoC - dbobj

18 years ago by thefirstofall@gmail.com — view source

unread

Hi!

I'm an interested student with a project idea for the PHP.net Google
Summer of Code.

I'm thinking about a native ( compiled from C ) ORM extension for PHP. I
don't need to explain how useful and important it is in modern web
applications / frameworks.

My goal is a native php extension that

is efficient / high performance (thanks to good application design
and native code)
has a really good API (thanks to PHP5's extensible object modell)
works with all major DBMSs (by using PDO)
is flexible, so with thin wrapper classes it could be a drop-in
replacement for any framework's ORM module.

The concept is object persistence (somewhat similar to Hibernate for
Java). I think object persistence is the most simple, clean and usable
ORM concept, however it is the hardest to implement especially in pure
PHP code (no access to PHP5's internal object modell).

So what do you think? Is it a good project for SoC?

About me: I'm 19 year old, a student at the Budapest University of
Technology and Economics on the IT department ... and as you have
already discovered, not a native English speaker.

Adam Banko

18 years ago by Jacob Santos — view source

unread

Hey Adam,

I think this project would be great for PHP, and hopefully upon stable release
it would be better than the current Great Satan (Hail Satan! Glory be upon his
name and wretched be his victims) that currently rules the ORM world.

It is one exercise that I looked into to learn PHP extension building, but I
ventured too far into the dark forest of PDO and fled for fear of my mortal
soul. Reflection extension is a like a black hole for sucking in n00b souls who
dare try to comprehend its mass l33tness.

(The above might come off as sarcasm, but I'm totally serious. Only Gods of man
would be able to maintain the aforementioned extensions or at least ones with
more experience with C.)

If you get it started, I would like to help you maintain it after the Summer of
Code project. I have other projects which would take precedence, but I think
this would make PHP even better.

You don't have to work directly with PDO (as I had thought), just do as SDO
does, and allow for a PDO object to be passed through the ORM constructor or
method. But you probably already knew this.

Hey, good luck getting accepted and to the completion of the project.

Jacob Santos

Hi!

I'm an interested student with a project idea for the PHP.net Google
Summer of Code.

I'm thinking about a native ( compiled from C ) ORM extension for PHP. I
don't need to explain how useful and important it is in modern web
applications / frameworks.

My goal is a native php extension that

is efficient / high performance (thanks to good application design
and native code)

has a really good API (thanks to PHP5's extensible object modell)

works with all major DBMSs (by using PDO)

is flexible, so with thin wrapper classes it could be a drop-in
replacement for any framework's ORM module.

The concept is object persistence (somewhat similar to Hibernate for
Java). I think object persistence is the most simple, clean and usable
ORM concept, however it is the hardest to implement especially in pure
PHP code (no access to PHP5's internal object modell).

So what do you think? Is it a good project for SoC?

About me: I'm 19 year old, a student at the Budapest University of
Technology and Economics on the IT department ... and as you have
already discovered, not a native English speaker.

Adam Banko

18 years ago by banko.adam@gmail.com — view source

unread

Hi!

I've created a draft of what would be the Abstract of my application.

Please review it, if I'm mistaken in something or left out an important
point. This will be the public part of the application. There will be a
more detailed description, but it will be visible only to mentors.

BEGIN

Creating a native object persistence module for PHP

Object persistence is a form of Object-Relational Mapping, a technology
that has recently became very important for complex web applications.
For the PHP developer, ORM means a layer that abstracts the complexity
of the relational database, and presents a nicer, object-oriented
interface. So it simplifies data access.

There are many different implementations, most are written in PHP. I'm
proposing a native (compiled from c) extension that would only support
some basic features, but can be extended (with PHP) to have complex,
high-level features (like database schema auto-discovery).

The overall goal is to have an extension bundled with PHP, that could be
a common base for full ORM solutions. If the currently very diverse ORM
implementations would have a common base, it would ease the learning of
different implementations, and enable ORM developers to concentrate on
high-level features. Using ORM solutions based on a native module would
also mean a smaller overhead compared to pure-PHP implementations.

The project (in it's current form) is about 70% ready, and during the
summer my plan is to make it about 90% complete.
The most important goals for the summer are:

Implementing a database driver module that uses PDO, so all PDO
supported databases will be supported.
Implementing exception-based error handling.
Rewriting the outer interface to what the community finds the best.
Some testing, finding and eliminating bugs.

So the extension will be mostly usable. However it won't be complete. It
will still lack:

A good documentation,
a full code review to eliminate possible security holes,
support for safe_mode and for cases where non-controlled PHP code is
used (for example web hosting).

For the current status of the project and more information consult the
project's wiki at http://dbobj.sourceforge.net/ .

END

Adam

18 years ago by thefirstofall@gmail.com — view source

unread

Hi!

I've created a draft of what would be the Abstract of my application.

Please review it, if I'm mistaken in something or left out an important
point. This will be the public part of the application. There will be a
more detailed description, but it will be visible only to mentors.

BEGIN

Creating a native object persistence module for PHP

Object persistence is a form of Object-Relational Mapping, a technology
that has recently became very important for complex web applications.
For the PHP developer, ORM means a layer that abstracts the complexity
of the relational database, and presents a nicer, object-oriented
interface. So it simplifies data access.

There are many different implementations, most are written in PHP. I'm
proposing a native (compiled from c) extension that would only support
some basic features, but can be extended (with PHP) to have complex,
high-level features (like database schema auto-discovery).

The overall goal is to have an extension bundled with PHP, that could be
a common base for full ORM solutions. If the currently very diverse ORM
implementations would have a common base, it would ease the learning of
different implementations, and enable ORM developers to concentrate on
high-level features. Using ORM solutions based on a native module would
also mean a smaller overhead compared to pure-PHP implementations.

The project (in it's current form) is about 70% ready, and during the
summer my plan is to make it about 90% complete.
The most important goals for the summer are:

Implementing a database driver module that uses PDO, so all PDO
supported databases will be supported.
Implementing exception-based error handling.
Rewriting the outer interface to what the community finds the best.
Some testing, finding and eliminating bugs.

So the extension will be mostly usable. However it won't be complete. It
will still lack:

A good documentation,
a full code review to eliminate possible security holes,
support for safe_mode and for cases where non-controlled PHP code is
used (for example web hosting).

For the current status of the project and more information consult the
project's wiki at http://dbobj.sourceforge.net/ .

END

Adam

18 years ago by thefirstofall@gmail.com — view source

unread

Hi!

First a small abstract:
Lukas suggested that a full ORM shouldn't be implemented in C. The short
reply is that I think that core ORM features could be implemented in C,
and that would be good.

Why? Read the rest..

I've looked into the phpdbabstraction list you liked and I think it's a
brilliant idea. I'm sure it'll generate a lot of innovation, and help
make the "PHP ORM world" better :)

I'll write to the list when I've gathered my thoughts, but for the time
I'll keep the main discussion here. This list is broader, and I'm
interested in the opinion of other, not only ORM developers.

Now, for some real issues. You said you think that "a full ORM belongs
into user space and not into C code". I'm thinking about pros and cons,
first for pure PHP and pure C implementation.

Pure PHP code is easier to write, easier to debug, and easier to
maintain than C code. So more and better features can be implemented in
less time.

Pure C code has access to PHP5's low level object API, so it can produce
a really intuitive interface (object persistence), that you can't do
from PHP code. C code uses a bit less memory and a lot less CPU time
(accessing a member in a struct vs. accessing a member in a PHP array).
A native extension can cache lot of hard-to-generate data in the process
memory (that doesn't get destroyed after requests).

So, the straight conclusion is that we should have hybrid solution, that
has the strengths of both words. The arising question is where to draw
the line?

From what you have written, I think you are suggesting to keep C code
minimal. I can imagine that in the following way: There is this
extension and the userspace application registers hooks on it, so it has
access to the interface benefits of the internal object model. This
more-or-less solved the outer interface problem, but generated a bigger
problem, PHP's internal interface is not ment to be used by user
scripts, and mixing the normal and this direct OO API would result in an
over-complicated and hard to maintain code.

The __get and __set methods create some magical behavior, but for this
we'd need a lot of magic, and we know magic don't scale well.

The other issue is performance. For the ORM field I didn't find
operation that could be moved out to native code efficiently. Most
complex operations need a lot of data, and passing this data to a native
function, doing decoding and after it's processed the time needed for
recoding would eliminate the performance benefit. For example ADODB
could implement the resultset_to_array method in C, because there's
minimal overhead on fetching a row from a database resource (negative
compared to it's PHP equivalent).

This isn't gonna work well. We need something different. And this the
solution I'm suggesting.

Let there be defined a small set of features that all ORM should have.
This is what most people use most of the time. Let's implement this set
of features in native code, and make it very flexible and extensible, so
other features could be added to it from PHP code. This solution
performs nicely for common tasks, but could also have many - not so
commonly used - fancy features implemented in PHP. Of course these extra
features won't be as fast as the rest, but this way special needs could
also be addressed.

So the frequently used features are fast, and the less-used features are
slow, but there these extra features are easy to implement. I think this
is good.

And one more point. In C you can implement complex object oriented
structures (lot of classes with fancy virtual functions) with small
overhead, while in PHP this structure could mean a lot of overhead. For
example separate class for every column type, and separate object for
every persistent value is unimaginable in PHP, but sounds OK in C. This
means that although managing PHP code should be easier than managing C
code, thanks to the stronger structure in C, in the end it could turn
out to be just as manageable as it's imagined PHP equivalent.

And as conclusion, some pseudo code, I'd like to see working:

//class Person {DB mapping options, constructor, ...}
$a = new Person("Lukas"); // create new record
$a->fiend = $f = Person::get(15); // setting an FK field
$b = Person::get(15); //won't load it again, reference it (singleton)
//update $f == $b 's age, and set the modified bit on this field
$a->friend->age++;
//no explicit save, implicit save on destruction

Thanks for your attention, Adam

1. 18, vasárnap keltezéssel 01.07-kor Lukas Kahwe Smith ezt írta:

Bankó Ádám wrote:

I'm an interested student with a project idea for the PHP.net Google
Summer of Code.

I'm thinking about a native ( compiled from C ) ORM extension for PHP. I
don't need to explain how useful and important it is in modern web
applications / frameworks.

IMHO a full ORM belongs into user space and not into C code. I kind of
like the approach that ADODB did, which was taking an existing DBAL and
moving selected items that where bottlenecks into C space. Thereby
providing a drop in speed improvement, while keeping the C code to a
minimum.

As such I would prefer that this would be done in collaboration with an
existing or in development PHP ORM.

I recently opened a mailing list on exactly the top of database
abstraction in PHP that already has well over 30 subscribers and pretty
much all of the key ORM developers in the PHP space:
http://pooteeweet.org/blog/611

Might be a good idea to bring up your thoughts there as well and see if
there is a chance for collaboration with an existing userland ORM project.

regards,
Lukas

18 years ago by Lukas Kahwe Smith — view source

unread

Bankó Ádám wrote:

Pure C code has access to PHP5's low level object API, so it can produce
a really intuitive interface (object persistence), that you can't do
from PHP code. C code uses a bit less memory and a lot less CPU time

I would hope that the solution for "low level object API" would be that
we expose more and more of that to user space. Stuff like being able to
force the calling of the parent constructor come to mind, I am sure
there are other things more relevant to the topic at hand as well. But
yes, its a valid point.

The other issue is performance. For the ORM field I didn't find
operation that could be moved out to native code efficiently. Most
complex operations need a lot of data, and passing this data to a native
function, doing decoding and after it's processed the time needed for
recoding would eliminate the performance benefit. For example ADODB
could implement the resultset_to_array method in C, because there's
minimal overhead on fetching a row from a database resource (negative
compared to it's PHP equivalent).

Yeah, you are right .. it might be hard to find things that can be moved
to C space and vice versa. I do not know where to draw the line this
very second either. I just want to make sure that this option is explored.

I remember that Thies once said, the ultimate goal of the PHP language
would be to make everything fast enough in PHP so that everything can be
done in user space. All the extension should do is expose third party
libs. Of course this might only be a dream, but I think its one of those
dreams worth purseing .. even if you never make it.

regards,
Lukas

18 years ago by thefirstofall@gmail.com — view source

unread

I remember that Thies once said, the ultimate goal of the PHP language
would be to make everything fast enough in PHP so that everything can be
done in user space. All the extension should do is expose third party
libs. Of course this might only be a dream, but I think its one of those
dreams worth purseing .. even if you never make it.

I'm thinking about if PHP was fast, and extensions were only exposing
library interfaces, then PHP would be C/C++/Java/etc.

In my opinion - in every language - low level stuff should be
implemented efficiently (with low level tools), so the things building
on it can perform.

In the beginning, database API usage was an important part of most PHP
applications. Then came the smart/thin DB abstraction layers, and the
database access API was wrapped to be a bit nicer and more-or-less
database independent. Now the topic is ORM, that wraps all database
related things, to provide a nice OO interface. And further, these ORM
solutions are only a small part of the modern web development
frameworks.

These are a lot of layers, and I think we should reconsider what is
low-level and what is not. Some years ago ORM was by no means low-level.
But as things are getting more complex, I think we could say, that ORM
is a low-level building block of the near-future's web applications.

Adam

18 years ago by Lukas Kahwe Smith — view source

unread

Hi,

just to clarify .. from your comments I am quite hopeful in your proposal.

regards,
Lukas

18 years ago by Stanislav Malyshev — view source

unread

Pure C code has access to PHP5's low level object API, so it can produce
a really intuitive interface (object persistence), that you can't do
from PHP code. C code uses a bit less memory and a lot less CPU time

Just curious - which access do you need that PHP API doesn't give?

And one more point. In C you can implement complex object oriented
structures (lot of classes with fancy virtual functions) with small

What do you mean by virtual functions? If you mean C++ meaning, all
class methods in PHP are "virtual".

overhead, while in PHP this structure could mean a lot of overhead. For
example separate class for every column type, and separate object for
every persistent value is unimaginable in PHP, but sounds OK in C. This

Why would it be that different in C - you'd still have to define the
same classes and same objects and play by the engine rules?

Stanislav Malyshev, Zend Products Engineer
stas@zend.com http://www.zend.com/

18 years ago by thefirstofall@gmail.com — view source

unread

First, here's what I've done so far in the subject:
http://dbobj.sourceforge.net/bzr/dbobj/

And here's a quite dreary wiki feel free to update it...
http://dbobj.sourceforge.net/wiki/HomePage

However, there are some notes on these:
1, I didn't spend much time on documentation writing, because both the
API and the internal workings are still very flexible. But you can have
a look the two tests applications I use (dbobj.php, forum.php), to get
some idea of how things work.

2, I've only finished modularising the database layer last day. So the
current code is REALLY UNSTABLE, it'll probably segfault if you try it
out. However, I'd like to receive crash reports (preferably with some
debugging information).

3, The current version has really BAD error handling, I'll implement
some exception based thing in the future.

Secondly, the answers to your questions

Just curious - which access do you need that PHP API doesn't give?

I didn't see any way for a PHP class to nicely handle

$myobject->i++;

where MyClass::i doesn't exist, and this should be relayed to some magic
function like __set;

I also don't know how well __get and __set behaves with references.

What do you mean by virtual functions? If you mean C++ meaning, all
class methods in PHP are "virtual".

Yeah, I know that.

Why would it be that different in C - you'd still have to define the
same classes and same objects and play by the engine rules?

no doubt C is more complex, but it also gives more freedom (eg. for
caching the mapping configuration in memory)
In C I can implement all the structure I want (as long it's sane)
without much worrying about what performance cost it will have. I
wouldn't do the same in PHP. I'm talking about separate classes for
every database column type and abstract data type; separate object for
every value I get from the database, and present to the script. That's a
lot, and I don't think I would use this structure in PHP.

So I wanted to point out, that C code could have nicer (more OO)
structure, then PHP code. And that good structure means good
manageability, even in a complex language like C (compared to PHP). Of
course only if it is accompanied by good documentation.

18 years ago by Guilherme Blanco — view source

unread

Hi,

This is an interesting project!
I am currently implementing an ORM tool based on this white paper from
Mr. Scott W. Ambler (from IBM):
http://www.ambysoft.com/essays/persistenceLayer.html

I think that a compiled C ORM tool doesn't have too much advantage
over a PHP code based. The performance impact can be so slow that can
disapoint you. But... a bundled ORM tool for PHP would be really
interesting!

If you are really interested, I can share my current implementation
and create a Google Code Project.

AFAIK, the best ORM tool for PHP is Doctrine: http://www.phpdoctrine.net/trac

I am currently on a travel and you'll probably notice a delay in my messages.

Best regards,

Pure C code has access to PHP5's low level object API, so it can produce
a really intuitive interface (object persistence), that you can't do
from PHP code. C code uses a bit less memory and a lot less CPU time

Just curious - which access do you need that PHP API doesn't give?

And one more point. In C you can implement complex object oriented
structures (lot of classes with fancy virtual functions) with small

What do you mean by virtual functions? If you mean C++ meaning, all
class methods in PHP are "virtual".

overhead, while in PHP this structure could mean a lot of overhead. For
example separate class for every column type, and separate object for
every persistent value is unimaginable in PHP, but sounds OK in C. This

Why would it be that different in C - you'd still have to define the
same classes and same objects and play by the engine rules?

Stanislav Malyshev, Zend Products Engineer
stas@zend.com http://www.zend.com/

--

--
Guilherme Blanco - Web Developer
CBC - Certified Bindows Consultant
Cell Phone: +55 (16) 9166-6902
MSN: guilhermeblanco@hotmail.com
URL: http://blog.bisna.com
São Carlos - SP/Brazil

18 years ago by thefirstofall@gmail.com — view source

unread

Hello!

This paper seems to be interesting, but I don't have the time to read it
now. If only there were such nice documentations when I started this!

On the performace point, I can't say anything well-grounded since I have
no benchmark results. I'll do some when I have a little more time, then
we'll see.

The other thing you mentioned ... the bundled ORM tool. I think it would
be good to have a "standard sollution" over the current diversity.

An analogy came to my mind ... session handling and object persistence.
Of course session handling is a much simpler task, I think they share
some common points. Like both can be implemented in a number of ways.
For session handling now there's a standard implementation that works
good for allmost all cases.

I don't say, that a similar thing can be achieved for ORM, nor do I say
that there should be one ORM implementation that everyone must use.
However I think that currently the ORM palette is too diverged (like if
everyone would use their own session implementation). There would be
good, if there were some "standard" thing, that fit's most people's
needs.

The bad thing is AFAIK that ORM is interpreted in many ways by PHP
developers, so there can't be a one-fits-all solution. For example there
are the structured programmer, who don't want to here about objects and
classes, but want a nicer interface to access their data in their
database. Taking these people into account when building an ORM is very
important if we want to keep PHP an easy to learn language.

Than there are the people who do everything object oriented, want some
of their objects to be persistent, and (for some reason) want them to be
stored in a relational database.

And there a lot of people in between, who would use an ORM solution
mixed with a lot of SQL.

I only mentioned three use cases, but in reality, I'm sure that the
image is much more complex.

However I believe that a "common divisor" could be found, something
common in all these cases, that can be made into some form of standard.
This shouldn't have many features, but should be extensible.

I think if the different more developer-friendly solutions were built on
"common divisor", it would result in better high level features, a nicer
learning curve (don't have to start from 0 when you need to learn a new
ORM implementation) and so on and so on.

What I'm really talking about, in the OO aspect, is a base class that
provides only some very basic (but very important) ORM features. It
should be small, quick and extensible. It should also have a really
good, intuitive interface.

What you think?

1. 19, hétfő keltezéssel 16.44-kor Guilherme Blanco ezt írta:

Hi,

This is an interesting project!
I am currently implementing an ORM tool based on this white paper from
Mr. Scott W. Ambler (from IBM):
http://www.ambysoft.com/essays/persistenceLayer.html

I think that a compiled C ORM tool doesn't have too much advantage
over a PHP code based. The performance impact can be so slow that can
disapoint you. But... a bundled ORM tool for PHP would be really
interesting!

If you are really interested, I can share my current implementation
and create a Google Code Project.

AFAIK, the best ORM tool for PHP is Doctrine: http://www.phpdoctrine.net/trac

I am currently on a travel and you'll probably notice a delay in my messages.

Best regards,

18 years ago by Jacob Santos — view source

unread

I agree that the optimal solution would be a base ORM solution that all others
can build off of and use. Just like PDO, it doesn't have to be an end all
solution. Just implement the major features that all ORM projects have.

The mailing list is great to pick at the brains that have been building such
solutions for years. Coming to a solution for a common API would be better in C
as a PHP extension. It would also be easy to check for the existence of the
extension to use it.

Jacob Santos

Hello!

This paper seems to be interesting, but I don't have the time to read it
now. If only there were such nice documentations when I started this!

On the performace point, I can't say anything well-grounded since I have
no benchmark results. I'll do some when I have a little more time, then
we'll see.

The other thing you mentioned ... the bundled ORM tool. I think it would
be good to have a "standard sollution" over the current diversity.

An analogy came to my mind ... session handling and object persistence.
Of course session handling is a much simpler task, I think they share
some common points. Like both can be implemented in a number of ways.
For session handling now there's a standard implementation that works
good for allmost all cases.

I don't say, that a similar thing can be achieved for ORM, nor do I say
that there should be one ORM implementation that everyone must use.
However I think that currently the ORM palette is too diverged (like if
everyone would use their own session implementation). There would be
good, if there were some "standard" thing, that fit's most people's
needs.

The bad thing is AFAIK that ORM is interpreted in many ways by PHP
developers, so there can't be a one-fits-all solution. For example there
are the structured programmer, who don't want to here about objects and
classes, but want a nicer interface to access their data in their
database. Taking these people into account when building an ORM is very
important if we want to keep PHP an easy to learn language.

Than there are the people who do everything object oriented, want some
of their objects to be persistent, and (for some reason) want them to be
stored in a relational database.

And there a lot of people in between, who would use an ORM solution
mixed with a lot of SQL.

I only mentioned three use cases, but in reality, I'm sure that the
image is much more complex.

However I believe that a "common divisor" could be found, something
common in all these cases, that can be made into some form of standard.
This shouldn't have many features, but should be extensible.

I think if the different more developer-friendly solutions were built on
"common divisor", it would result in better high level features, a nicer
learning curve (don't have to start from 0 when you need to learn a new
ORM implementation) and so on and so on.

What I'm really talking about, in the OO aspect, is a base class that
provides only some very basic (but very important) ORM features. It
should be small, quick and extensible. It should also have a really
good, intuitive interface.

What you think?

18 years ago by Tony Bibbs — view source

unread

Lukas Kahwe Smith wrote:

IMHO a full ORM belongs into user space and not into C code. I kind of
like the approach that ADODB did, which was taking an existing DBAL and
moving selected items that where bottlenecks into C space. Thereby
providing a drop in speed improvement, while keeping the C code to a
minimum.

I agree. I'm a user of Propel (http://propel.phpdb.org) which is in the
user space as you suggest and I think it works fine. The current beta
version of Propel uses PDO so execution speed isn't really a huge
problem. I think the only thing worth considering adding to the C-code
is something to compliment PDO that can collect all the database
metadata that ORM's require. Propel currently does this using Creole
(one of the many DB abstraction libraries out there).

As such I would prefer that this would be done in collaboration with an
existing or in development PHP ORM.

Again, I agree. And being a bit biased I'd simply like to throw Propel
into the mix for consideration.

--Tony

18 years ago by Lukas Kahwe Smith — view source

unread

Tony Bibbs wrote:

I agree. I'm a user of Propel (http://propel.phpdb.org) which is in the
user space as you suggest and I think it works fine. The current beta
version of Propel uses PDO so execution speed isn't really a huge

FYI: You are mistaken if you think that moving from the old extensions
to PDO provides a speed improvement (there is rather decrease unless you
are using fetchAll()).

problem. I think the only thing worth considering adding to the C-code
is something to compliment PDO that can collect all the database
metadata that ORM's require. Propel currently does this using Creole

And this imho is the thing that should definitely stay in userland. Its
a giant hackery to get it to work. This is something people need to be
able to adapt quickly. Putting this in C would be a maintenance nightmare.

regards,
Lukas

18 years ago by Tony Bibbs — view source

unread

Lukas Kahwe Smith wrote:

FYI: You are mistaken if you think that moving from the old extensions
to PDO provides a speed improvement (there is rather decrease unless you
are using fetchAll()).

Sorry, I wasn't clear on this. Creole is the DB abstraction layer on top
of the "old extensions". The older versions of Propel were implemented
that way. The new beta version uses PDO dicrectly (i.e. minus Creole)
and that showed significant speed improvements over prior versions.

And this imho is the thing that should definitely stay in userland. Its
a giant hackery to get it to work. This is something people need to be
able to adapt quickly. Putting this in C would be a maintenance nightmare.

I could see that. Just wanted to float the idea out there for discussion.

--Tony

18 years ago by Lukas Kahwe Smith — view source

unread

Tony Bibbs wrote:

Sorry, I wasn't clear on this. Creole is the DB abstraction layer on top
of the "old extensions". The older versions of Propel were implemented
that way. The new beta version uses PDO dicrectly (i.e. minus Creole)
and that showed significant speed improvements over prior versions.

Ah, that must be a Creole issue then .. because there is no technical
reason for this.

regards,
Lukas

18 years ago by thefirstofall@gmail.com — view source

unread

1. 19, hétfő keltezéssel 16.51-kor Richard Lynch ezt írta:

I don't think you'd want the RAM/performance hit of that, even in C...
[shrug]

OTOH, I don't think I'd even want to use an ORM, so feel free to
ignore me completely. :-)

Hi!

I think a code is more manageable if it's more structured. This
additional structure however means "unoptimal" code, that will run
slower, and consume more memory.

Some balance point should be found, where the project is still
manageable, but the resource overhead is minimal.

Managing C code with limited structure (for bigger projects than hello
word) is hell. This is why structured programming and then the OOP was
invented.

Having a separate object for every data value is a bit idealistic (it
would be best, if it wouldn't have the object construction / destruction
overhead. But it has.

I choose to implement this "idealistic" approach when I started to see
switch-case blocks in my code, that did not fit in my screen (about 50
lines).

Adam

18 years ago by Guilherme Blanco — view source

unread

Hi,

I'd like to help you in this working draft, but I don't have too much
experience with C/C++ to holp with code, but... I can try something.

Also, I suggested you that white paper since it supports almost
everything expected from a robust ORM tool. I still believe that your
performance gain will be less than expected, but I still like the
idea. I even spoke with Mr. Manuel Lemos (PHPClasses.org author)
almost an year ago in a PHP event here in Brazil exactly about the
same thing. Our conclusion reached that the performance would not
justify the implementation.

But, I still believe this is an important feature to be bundled with
PHP and, if not, at east be a separated supported project of PHP (like
Smarty). Currently, in a MVC/MVP perspective, PHP already supports the
View layer, but misses the Model (sorry, they helped a lot with PDO
talking about DB Abstraction) layer.
I believe that PHP should have a bundled feature that can even support
OO approach for OO enthusiasts (like me) and also Structural. If not
bundled, support the project like Smarty, something like
http;//orm.php.net

As I mentioned before, the best ORM tool I've notice is Doctrine, but
it has some issues that I can list if necessary. The point is not to
start a discussion about why Propel, MetaStorage, ezComponents,
ADODB_ActiveRecord or any other is better or not; the point is that
different implementations has some good points and bad points. Tony
Bibbs already started a mailing list with already implemented ORM tool
authors, that can be a great starting point.

I have to say that your idea is good enough to even be out of Google
SoC. This could be a separate project, and should already exist for
years. If the only needed starting point is SoC, that's ok. But you
have to understand that this project should already exist in PHP land.
Java already has a lot of ORM tools, and Hibernate is the most known.
PHP should have a similar thing.

I am not part of Core Team, but I'll deliver my proposal to everyone here.
If we group ORM authors, generate a draft and implement it, I think we
can reach a commom sense. As I already mentioned, this is out of SoC
scope.

Please, do not ignore the idea. An answer that this does not belong to
PHP land is going against your own principles, since you're trying to
bring PHP more OO.

I am opened to start a dialog about this subject.
If you want to add me to your MSN/GTalk/Yahoo!/Skype:
MSN: guilhermeblanco@hotmail.com
GTalk/Yahoo!/Skype: guilhermeblanco

Best regards,

19, hétfő keltezéssel 16.51-kor Richard Lynch ezt írta:

I don't think you'd want the RAM/performance hit of that, even in C...
[shrug]

OTOH, I don't think I'd even want to use an ORM, so feel free to
ignore me completely. :-)

Hi!

I think a code is more manageable if it's more structured. This
additional structure however means "unoptimal" code, that will run
slower, and consume more memory.

Some balance point should be found, where the project is still
manageable, but the resource overhead is minimal.

Managing C code with limited structure (for bigger projects than hello
word) is hell. This is why structured programming and then the OOP was
invented.

Having a separate object for every data value is a bit idealistic (it
would be best, if it wouldn't have the object construction / destruction
overhead. But it has.

I choose to implement this "idealistic" approach when I started to see
switch-case blocks in my code, that did not fit in my screen (about 50
lines).

Adam

--

--
Guilherme Blanco - Web Developer
CBC - Certified Bindows Consultant
Cell Phone: +55 (16) 9166-6902
MSN: guilhermeblanco@hotmail.com
URL: http://blog.bisna.com
São Carlos - SP/Brazil

18 years ago by Lukas Kahwe Smith — view source

unread

Guilherme Blanco wrote:

As I mentioned before, the best ORM tool I've notice is Doctrine, but
it has some issues that I can list if necessary. The point is not to

BTW: the lead author of Doctrine aready submitted an SoC project idea
for php.net on googles website.

regards,
Lukas

18 years ago by Jeff Moore — view source

unread

Hi,

Sorry, I'm coming to this discussion a bit late and perhaps I don't
understand the all issues, but I think the scope of creating an ORM
for PHP is probably too large. Let me suggest a smaller project that
could lead to better ORM.

PHP has several methods of serializing and unserializing objects to
different formats:

PHP serialization format using serialize/unserialize (with __sleep,
__wakeup and the Serializable interface)
PHP code using var_export and eval (with __set_state)
JSON using json_encode and json_decode
Partial support for DB Records using mysql_fetch_object or
PDOStatement->fetchObject()

Its safe to assume that other serialization formats are possible.

All of these methods have to address the same issues with objects:

How to handle protected and private properties?
To call or bypass the constructor on unserialization?
Can objects become involved with their own serialization? (such as
__sleep, __wake, __set_state, and Serializable)
Must objects become involved with their own serialization (such as
with var_export and __set_state) or can you serialize unaware objects?
Are object relationships and identity preserved? (only true for
serialize/unserialize right now)
How are classes loaded? (unserialize_callback_func and __autoload?)

I think a more limited scope project would be to attempt to address
these core object serialization issues across the multiple formats
that PHP already provides in some consistent way. Of course, not
every capability is possible or necessary with every format.

Before implementing ORM in C, which would be relatively less flexible
than a PHP implementation, perhaps it would be a good idea to profile
the existing PHP ORM solutions and look for opportunities to
implement key functions in C, while leaving the majority to user space.

A possible example of such an opportunity might be some equivalent to
extract() and compact() only for converting between arrays and object
properties instead of arrays and local variables.

Rather than implement a full blown ORM extension, perhaps
implementing some object serialization enhancements would be a good
alternative, or at least a good first step? Perhaps some user space
ORM profiling would be a good way to find the second step?

Best Regards,

Jeff

18 years ago by Adam Banko — view source

unread

1. 27, kedd keltezéssel 16.55-kor Jeff Moore ezt írta:

Hi,

Sorry, I'm coming to this discussion a bit late and perhaps I don't
understand the all issues, but I think the scope of creating an ORM
for PHP is probably too large. Let me suggest a smaller project that
could lead to better ORM.
Yes a bit late ... as the application submission deadline has already
passed. (for SoC)

PHP has several methods of serializing and unserializing objects to
different formats:

PHP serialization format using serialize/unserialize (with __sleep,
__wakeup and the Serializable interface)
PHP code using var_export and eval (with __set_state)
JSON using json_encode and json_decode
Partial support for DB Records using mysql_fetch_object or
PDOStatement->fetchObject()

Its safe to assume that other serialization formats are possible.

All of these methods have to address the same issues with objects:

How to handle protected and private properties?
To call or bypass the constructor on unserialization?
Can objects become involved with their own serialization? (such as
__sleep, __wake, __set_state, and Serializable)
Must objects become involved with their own serialization (such as
with var_export and __set_state) or can you serialize unaware objects?
Are object relationships and identity preserved? (only true for
serialize/unserialize right now)
How are classes loaded? (unserialize_callback_func and __autoload?)

I think a more limited scope project would be to attempt to address
these core object serialization issues across the multiple formats
that PHP already provides in some consistent way. Of course, not
every capability is possible or necessary with every format.

The SoC application is over, but this could also be done outside of SoC.
I think this is a good project, rethinking the serialization /
unserialization support in PHP would be beneficial for many. However
don't wait for me to do it, because I won't.

Before implementing ORM in C, which would be relatively less flexible
than a PHP implementation, perhaps it would be a good idea to profile
the existing PHP ORM solutions and look for opportunities to
implement key functions in C, while leaving the majority to user space.

I think I've already explained my opinion on this idea. Your idea is not
bad, and I think many agrees with you, but I see things a bit
differently.

A possible example of such an opportunity might be some equivalent to
extract() and compact() only for converting between arrays and object
properties instead of arrays and local variables.

I generally don't like the idea of converting an object to array and
back as it violates most OO principles. Don't convert an object to an
array, because than there's no problem with private properties and so
on. If someone wants arrays from the database, then he/she souldn't use
ORM because it's Object-relational mapping, not Array-relational
mapping!

Rather than implement a full blown ORM extension, perhaps
implementing some object serialization enhancements would be a good
alternative, or at least a good first step? Perhaps some user space
ORM profiling would be a good way to find the second step?
Maybe ... but that is someone else's project, not mine. To see what my
goals are how I plan to achieve them, see the archives. I think I've
explained a lot on the list.

However, if you find any problems with my proposed solution, please
don't hesitate to share it with us.

Best Regards,

Jeff

Thank you,
Adam

18 years ago by thefirstofall@gmail.com — view source

unread

I believe that PHP should have a bundled feature that can even support
OO approach for OO enthusiasts (like me) and also Structural. If not
bundled, support the project like Smarty, something like
http://orm.php.net
...
I have to say that your idea is good enough to even be out of Google
SoC. This could be a separate project, and should already exist for
years. If the only needed starting point is SoC, that's ok. But you
have to understand that this project should already exist in PHP land.
Java already has a lot of ORM tools, and Hibernate is the most known.
PHP should have a similar thing.

I like the idea of the bundled ORM support, I think it's getting so
common, that bundling it with PHP makes sense. However, in the C vs. PHP
question I agree with Jacob Santos, I think PHP land implementations are
good, but if we create a common API, it should be written in C. Why is
PDO implemented as a native extension?

The project is existing, I'm doing it for about a year and a half, and
SoC is way I can spend more time on it in the summer.

I am not part of Core Team, but I'll deliver my proposal to everyone here.
If we group ORM authors, generate a draft and implement it, I think we
can reach a commom sense. As I already mentioned, this is out of SoC
scope.
I don't see why any open source project should be out of SoC scope. It's
only a way for students to work on OS project in the summer instead of
flipping burgers in McDonald's.

Please, do not ignore the idea. An answer that this does not belong to
PHP land is going against your own principles, since you're trying to
bring PHP more OO.

Yes, I like OO, and want create good object-oriented API, but I don't
see why can't that be achieved with a native extension. Maybe you are
thinking that C is not an OO language, so I can't build nicely object
oriented software with it? If you look at a source you can see that OO
and C can live together nicely.
http://sourceforge.net/project/showfiles.php?group_id=183889

I am opened to start a dialog about this subject.
If you want to add me to your MSN/GTalk/Yahoo!/Skype:
MSN: guilhermeblanco@hotmail.com
GTalk/Yahoo!/Skype: guilhermeblanco

I've added you to my contact list, but I think the main discussion
belongs to this list.

Adam

18 years ago by Robin Ericsson — view source

unread

The project is existing, I'm doing it for about a year and a half, and
SoC is way I can spend more time on it in the summer.

If there is someone willing to something, and someone else is paying
for it, let him do it. Why should it bother whether it's C or PHP? The
community will benefit from it either way.

--
regards,
Robin

17 years ago by Lukas Kahwe Smith — view source

unread

Hello Banko,

I remember that you proposed an ORM written in C/C++ last year around
the same time. Back then I was quite opposed to the idea, saying that
something like this best belongs in userland, and that at most certain
bottleneck features should be moved to C (following the example of the
ADODB optional companion extension). I still believe this to be true.

However my opinions back then were a bit short sighted, as I focused
on the fact that a proper ORM needs a proper DBAL underneath to really
work. As such I feel that PDO does not fulfill this requirement, and
actually I think it never will/should do this. Like I stated back
then, database sql abstraction is ugly, tedious and hacky. This is
something the user should be able to easily and quickly adapt to his
needs. However one line of thought I ignored back then (and I do not
remember anyone pointing this out), is that of course such an ORM
could leverage a userland DBAL that follows a defined interface. I
think such an approach could indeed be feasible. Then again writing an
ORM in C is a huge undertaking and maybe again the best approach would
be to pick an existing ORM and start a companion extension that one by
one replaces the bottleneck areas with a C implementation. One day
this might lead to a full fledged ORM in C, but in the mean time it
already provides real world benefits.

regards,
Lukas

18 years ago by thefirstofall@gmail.com — view source

unread

Its a question of maintainability. Stuff like reverse engineering
schema's from a database is simply not sensible to be done by C code. It
requires a low barrier to entry, the ability to quickly fix things if
you encounter a newer or very old obscure RDBMS version etc.

Doing database schema discovery is a thing that should be implemented in
PHP, and should never be considered a "core" ORM feature.

What I've done is a callback hook in the system, that let's the PHP
script get this data (from schema discovery, an XML file or anything
else).

Maintainability is an important point, and I've been trying to build a
manageable code. The many class/object concept is mainly for this.
I think most of the code is well structured and with a well-written
documentation maintainability can be achieved.

Adam

18 years ago by Andrey Hristov — view source

unread

Hi,
Lukas Kahwe Smith wrote:

Robin Ericsson wrote:

The project is existing, I'm doing it for about a year and a half, and
SoC is way I can spend more time on it in the summer.

If there is someone willing to something, and someone else is paying
for it, let him do it. Why should it bother whether it's C or PHP? The
community will benefit from it either way.

Its a question of maintainability. Stuff like reverse engineering
schema's from a database is simply not sensible to be done by C code. It
requires a low barrier to entry, the ability to quickly fix things if
you encounter a newer or very old obscure RDBMS version etc.

Then make a mix of PHP and C code. C call call PHP userland, so it
shouldn't be a problem. And as I see it, it is always good to have
reference implementation in PHP and port it to C. I think Marcus did it
while implementing SPL.

regards,
Lukas

Ciao,
Andrey

18 years ago by Lukas Kahwe Smith — view source

unread

Andrey Hristov wrote:

Hi,
Lukas Kahwe Smith wrote:

Robin Ericsson wrote:

The project is existing, I'm doing it for about a year and a half, and
SoC is way I can spend more time on it in the summer.
If there is someone willing to something, and someone else is paying
for it, let him do it. Why should it bother whether it's C or PHP? The
community will benefit from it either way.
Its a question of maintainability. Stuff like reverse engineering
schema's from a database is simply not sensible to be done by C code. It
requires a low barrier to entry, the ability to quickly fix things if
you encounter a newer or very old obscure RDBMS version etc.

Then make a mix of PHP and C code. C call call PHP userland, so it
shouldn't be a problem. And as I see it, it is always good to have
reference implementation in PHP and port it to C. I think Marcus did it
while implementing SPL.

Yes .. so for the proposal .. step 1) would be defining interfaces and
abstract classes to represent things. this could go into C

regards,
Lukas

18 years ago by Jacob Santos — view source

unread

Developing the extension like SPL would allow for very quick development and
maintainability for the extension. It is extremely easy to have interfaces,
exceptions, and in some cases abstract classes written in C and easily updated
when changes, if any, are made.

I was able to translate a great portion of one project library to a PHP
extension over a short time period of three days (of course I had the Extending
PHP book, PDO, and SPL as references... and luckily I didn't need to work with
any of the C Standard Library).

The way the current project is handled is extremely good. It shouldn't be
difficult to isolate where the changes need to be made and make them.

As for maintainability, if there are enough programmers, then it shouldn't be a
hassle to maintain the extension. There are quite a number of features that
should be implemented in C, even if they are extremely difficult, because it
would take too long to process in PHP.

I think the ORM C PHP extension will be quicker than anything in PHP. It doesn't
have to handle everything, just the most common ORM features and allow
extending. Just need to decide what features that is and what features the
project already has.

I do think it would be easier to come up with a standard for the project, if
perhaps Banko didn't spend a whole year and a half on it. I do hope that the
mailing list sorts out which features should be in the PHP extension to allow
for interoperability between the ORM PHP libraries. It would probably take
another year, but it would be nice if it was bundled in a later version of PHP
5.x and PHP 6.

Jacob Santos

Lukas Kahwe Smith wrote:

Andrey Hristov wrote:

Hi,
Lukas Kahwe Smith wrote:

Robin Ericsson wrote:

The project is existing, I'm doing it for about a year and a half, and
SoC is way I can spend more time on it in the summer.
If there is someone willing to something, and someone else is paying
for it, let him do it. Why should it bother whether it's C or PHP? The
community will benefit from it either way.
Its a question of maintainability. Stuff like reverse engineering
schema's from a database is simply not sensible to be done by C code. It
requires a low barrier to entry, the ability to quickly fix things if
you encounter a newer or very old obscure RDBMS version etc.

Then make a mix of PHP and C code. C call call PHP userland, so it
shouldn't be a problem. And as I see it, it is always good to have
reference implementation in PHP and port it to C. I think Marcus did it
while implementing SPL.

Yes .. so for the proposal .. step 1) would be defining interfaces and
abstract classes to represent things. this could go into C

regards,
Lukas

18 years ago by thefirstofall@gmail.com — view source

unread

I do think it would be easier to come up with a standard for the project, if
perhaps Banko didn't spend a whole year and a half on it. I do hope that the
mailing list sorts out which features should be in the PHP extension to allow
for interoperability between the ORM PHP libraries. It would probably take
another year, but it would be nice if it was bundled in a later version of PHP
5.x and PHP 6.

Maybe true, it would be easier to create the standard from scratch. But
look at the good side. There's a current API that we can start talking
about. The current outer interface (what matters most for PHP
developers) is very flexible. And if the community can settle at a
standard API, I'm willing to implement it.

Please, everyone, have a look at one of the examples (dbobj.php or
forum.php in the source) and decide if the current API is a good
starting pont.

As for the SoC application, I'll try to write something not too specific
so the specification won't go against the community's future decisions.
I hope google will accept it this way...

Adam

Lukas Kahwe Smith wrote:

Andrey Hristov wrote:

Hi,
Lukas Kahwe Smith wrote:

Robin Ericsson wrote:

The project is existing, I'm doing it for about a year and a half, and
SoC is way I can spend more time on it in the summer.
If there is someone willing to something, and someone else is paying
for it, let him do it. Why should it bother whether it's C or PHP? The
community will benefit from it either way.
Its a question of maintainability. Stuff like reverse engineering
schema's from a database is simply not sensible to be done by C code. It
requires a low barrier to entry, the ability to quickly fix things if
you encounter a newer or very old obscure RDBMS version etc.

Then make a mix of PHP and C code. C call call PHP userland, so it
shouldn't be a problem. And as I see it, it is always good to have
reference implementation in PHP and port it to C. I think Marcus did it
while implementing SPL.

Yes .. so for the proposal .. step 1) would be defining interfaces and
abstract classes to represent things. this could go into C

regards,
Lukas

18 years ago by Andrey Hristov — view source

unread

Hi,
Bankó Ádám wrote:

Then make a mix of PHP and C code. C call call PHP userland, so it
shouldn't be a problem.

I don't want to write a full ORM, only a common base, that others can
extend .. for example with INI/XML file configuration, schema auto
discovery etc.

And as I see it, it is always good to have a reference implementation
in PHP and port it to C. I think Marcus did it while implementing SPL.

If I'd be doing it Top-down, that clearly would be a good idea. But I'm
doing it with a iterative, XP like method.

you can do the same with XP too. As in PHP so in C you will have to
establish a common ground on which the further development will stand
on. Then part by part implement in PHP and port to C. Having tests will
help a lot.

First I do simple implementations for parts, and then I extend it to
support more features. For example on the first run, the system was
monolithic, now it has database drivers, database types and high-level
types implemented as sub-modules (mostly as polymorphic classes).

This is not the most effective approach, but it produces working code
early.

Adam

Andrey

18 years ago by thefirstofall@gmail.com — view source

unread

you can do the same with XP too. As in PHP so in C you will have to
establish a common ground on which the further development will stand
on. Then part by part implement in PHP and port to C. Having tests will
help a lot.

Maybe you are right, but I don't like the idea of doing everything
twice. Once for the PHP and once for the C version. I think for me this
would more disadvantages then advantages.

Adam

google SoC - dbobj

Why would it be that different in C - you'd still have to define the same classes and same objects and play by the engine rules?

Why would it be that different in C - you'd still have to define the same classes and same objects and play by the engine rules?

Why would it be that different in C - you'd still have to define the
same classes and same objects and play by the engine rules?

Why would it be that different in C - you'd still have to define the
same classes and same objects and play by the engine rules?