Implemented fallocate() syscall in streams

Hi Julien,

Hi,

I just wrote a patch to add fallocate() syscall support for streams.
It relies on posix_fallocate(), so that it should support many Unixes.
Linux's got a specification with a fallocate() function, more powerful
than the posix call.

Nice!

I propose two APIs for PHP :

• One that adds a function : fallocate()
  https://github.com/jpauli/php-src/tree/fallocate

• One that relies on ftruncate() , and adds a <bool>$use_fallocate
  flag https://github.com/jpauli/php-src/tree/fallocate_flag

Please, note that the latest proposal requires patches in different
extensions, as I changed a PHP_API function signature.

I'd prefer the first option.

I don't know if Windows can support that. Pierre, Anatol ? :-)

Tests are beeing written at the moment.

I didn't implement this is user stream handlers, as this really is a
low level implementation design that is kind of useless for usage in user
streams.

Thoughts ?

I've found this code

http://hg.mozilla.org/mozilla-central/file/3d846420a907/xpcom/glue/FileUtils.cpp#l61

Very good catch!

--
Regards,
Mike

Hi Julien,

Hi,

I just wrote a patch to add fallocate() syscall support for streams.
It relies on posix_fallocate(), so that it should support many Unixes.
Linux's got a specification with a fallocate() function, more powerful
than the posix call.

fallocate() does write blocks to the underlying filesystem (not so many
are supported, but the ones not supported should proxy to ftruncate()) and
prevents sparse file creation whereas ftruncate() just updates inode
information, leading to possible future out of space errors.

I got a POC showing the diffs between both calls here :
https://gist.github.com/jpauli/8afec7c4fc2b38f8ff27

I propose two APIs for PHP :

• One that adds a function : fallocate()
  https://github.com/jpauli/php-src/tree/fallocate

• One that relies on ftruncate() , and adds a <bool>$use_fallocate
  flag https://github.com/jpauli/php-src/tree/fallocate_flag

Please, note that the latest proposal requires patches in different
extensions, as I changed a PHP_API function signature.

I don't know if Windows can support that. Pierre, Anatol ? :-)

Tests are beeing written at the moment.

I didn't implement this is user stream handlers, as this really is a
low level implementation design that is kind of useless for usage in user
streams.

Thoughts ?

I've found this code

http://hg.mozilla.org/mozilla-central/file/3d846420a907/xpcom/glue/FileUtils.cpp#l61

On windows there's no such syscalls. All the data is however buffered, as
usual. An emulation could be of course implemented for Windows and other
unsupported platforms. However it probably wouldn't mimic the exact
behavior.

But besides writing in blocks (fallocate) and fast truncating - there are
some other useful things i see on the man page. I mean checking for the
free space or all those flags like zeroing file space. Actually, maybe
it'd be even more useful to put it into user space, maybe as normal
functions? Otherwise, if you think it's too low level, one could go two
ways - ether implement an emulation, or just use HAVE_FALLOCATE and a
configure switch to activate it manually.

You talk about fallocate() function.
My patch is based on posix_fallocate() call, which doesn' implement
all those (interesting) flags.
fallocate() is Linux only , that's why we can't rely on it and can't
publish it in user land.

And for Windows, I guess we could write a compat implementation (like
the mozilla one), but it anyway won't do the same job

Julien.P

Hi Julien,

Hi,

I just wrote a patch to add fallocate() syscall support for streams.
It relies on posix_fallocate(), so that it should support many Unixes.
Linux's got a specification with a fallocate() function, more powerful
than the posix call.

fallocate() does write blocks to the underlying filesystem (not so many
are supported, but the ones not supported should proxy to ftruncate()) and
prevents sparse file creation whereas ftruncate() just updates inode
information, leading to possible future out of space errors.

I got a POC showing the diffs between both calls here :
https://gist.github.com/jpauli/8afec7c4fc2b38f8ff27

I propose two APIs for PHP :

• One that adds a function : fallocate()
  https://github.com/jpauli/php-src/tree/fallocate

• One that relies on ftruncate() , and adds a <bool>$use_fallocate
  flag https://github.com/jpauli/php-src/tree/fallocate_flag

Please, note that the latest proposal requires patches in different
extensions, as I changed a PHP_API function signature.

I don't know if Windows can support that. Pierre, Anatol ? :-)

Tests are beeing written at the moment.

I didn't implement this is user stream handlers, as this really is a
low level implementation design that is kind of useless for usage in user
streams.

Thoughts ?

I've found this code

http://hg.mozilla.org/mozilla-central/file/3d846420a907/xpcom/glue/FileUtils.cpp#l61

On windows there's no such syscalls. All the data is however buffered, as
usual. An emulation could be of course implemented for Windows and other
unsupported platforms. However it probably wouldn't mimic the exact
behavior.

But besides writing in blocks (fallocate) and fast truncating - there are
some other useful things i see on the man page. I mean checking for the
free space or all those flags like zeroing file space. Actually, maybe
it'd be even more useful to put it into user space, maybe as normal
functions? Otherwise, if you think it's too low level, one could go two
ways - ether implement an emulation, or just use HAVE_FALLOCATE and a
configure switch to activate it manually.

Regards

Anatol

Hey Anatol ;-)

Would you mind working with me finalizing this patch so that we could
merge it in master for a 5.7 and Next eventually ?
I'll update the PR next week with tests, then we'll be able to talk
about the implementation to use for crossplatform.

Thank you

Julien.P

On Tue, Jul 8, 2014 at 9:37 AM, Ivan Enderlin @ Hoa
ivan.enderlin@hoa-project.net wrote:

Hi,

Hello :-),

I just wrote a patch to add fallocate() syscall support for streams.
It relies on posix_fallocate(), so that it should support many Unixes.
Linux's got a specification with a fallocate() function, more powerful
than the posix call.

fallocate() does write blocks to the underlying filesystem (not so
many are supported, but the ones not supported should proxy to
ftruncate()) and prevents sparse file creation whereas ftruncate()
just updates inode information, leading to possible future out of
space errors.

I got a POC showing the diffs between both calls here :
https://gist.github.com/jpauli/8afec7c4fc2b38f8ff27

I propose two APIs for PHP :

• One that adds a function : fallocate()
  https://github.com/jpauli/php-src/tree/fallocate

I prefer this one.

• One that relies on ftruncate() , and adds a <bool>$use_fallocate
  flag https://github.com/jpauli/php-src/tree/fallocate_flag

Please, note that the latest proposal requires patches in different
extensions, as I changed a PHP_API function signature.

I don't know if Windows can support that. Pierre, Anatol ? :-)

Tests are beeing written at the moment.

I didn't implement this is user stream handlers, as this really is a
low level implementation design that is kind of useless for usage in
user streams.

Thoughts ?

If the first one is used, please, consider exposing it on the user-land of
stream wrappers (exemple: stream_allocate) if possible.

I find it very useless as user stream already have the ftruncate
exposed to them.
The difference between ftruncate and fallocate is meaningless in userland.

Julien.P