[Yafray-devel] Fork-fix/command-line patch

[Steve Smith]

Hi all,

I've just uploaded the RC version of a patch I've been working on to fix
the fork stuff and be able to enable it on the command line.

http://www.coala.uniovi.es/~jandro/noname/mantis/bug_view_page.php?bug_id=0000020

What's the chance of getting this merged into the main tree? I was
planning on adding MPI cluster support to Yafray soon and it would be
based on this patch. If it won't go in I'll just maintain a separate
tree for the time-being.

Cheers,
Steve/Tarka

[Alejandro Conty Estevez]

Post by Steve Smith
I've just uploaded the RC version of a patch I've been working on to fix
the fork stuff and be able to enable it on the command line.
http://www.coala.uniovi.es/~jandro/noname/mantis/bug_view_page.php?bug_id=0000020
What's the chance of getting this merged into the main tree? I was

If the patch works ok, or at least compiles ok, I'll merge it, no prob.
Will check as soon as possible.

Post by Steve Smith
planning on adding MPI cluster support to Yafray soon and it would be
based on this patch. If it won't go in I'll just maintain a separate
tree for the time-being.

For the MPI version meybe is better to keep a separate tree, since it
involves polluting too much code, right? We can put a cvs repository
for that and info in the main site if you like.

jandro

[Steve Smith]

Post by Alejandro Conty Estevez
If the patch works ok, or at least compiles ok, I'll merge it, no prob.
Will check as soon as possible.

Cool, thanks.

Post by Alejandro Conty Estevez
For the MPI version meybe is better to keep a separate tree, since it
involves polluting too much code, right? We can put a cvs repository
for that and info in the main site if you like.

Actually it should only be a case of adding a couple of files and
tweaking the makefile and configure scripts. The way I've done the
mono/fork/thread stuff is to use inheritance to keep a clear separation
of the functionality. With the latest patch the inheritance tree looks
like:

scene_t (mono)
|
--------------
| |
forkscene_t threadscene_t

With the MPI stuff it will probably look more like:


scene_t (mono)
|
--------------------
| |
multiproc_t (abstract) threadscene_t
|
--------------------
| |
forkscene_t mpiscene_t

(Hopefully these diagrams work)

multiproc_t contains common functionality (the actually child worker
code), while the descendants override the communication functions (ie.
what is currently in ipc.cc will go into forkscene_t, MPI specific stuff
will go into mpiscene_t.

Anyway, I'll work on the principle that the existing patch will go in
and produce a proof-of-concept of the MPI stuff and see how you feel
about including it.

Cheers,
Steve/Tarka

[Diego Pino Navarro]

Hi Steve,

Maybe my question is stupid(very), but do you think there is a way to
make yafray work under Apples Xgrid cluster app, using your
fork/thread modification + some coding? I mean splitting an single
render process in several independent threads or forks that xgrid can
effectively understand and that can be send to every node in apples
grid (and maybe as result get several outputs(images) that can be
batch joined together later?). I have just tested xgrid using
one/unique unix application over the network with different parameters
for each node, resulting in independent data results that can be joined
together later(math simulation), but it looks like xgrid can make use
of a threaded multi processor app and distribute each job to a
different node. I asked for this to Jandro on the web forum but he said
it would take a lot of coding. (by the way xgrid don´t use MPI)

I hope it´s possible, anyway, i will try to compile your yafray with
your patch on my G4 and give it a test.

Thank´s a lot



Diego Pino N

Post by Steve Smith

Post by Alejandro Conty Estevez
If the patch works ok, or at least compiles ok, I'll merge it, no
prob.
Will check as soon as possible.

Cool, thanks.

Post by Alejandro Conty Estevez
For the MPI version meybe is better to keep a separate tree, since it
involves polluting too much code, right? We can put a cvs repository
for that and info in the main site if you like.

Actually it should only be a case of adding a couple of files and
tweaking the makefile and configure scripts. The way I've done the
mono/fork/thread stuff is to use inheritance to keep a clear separation
of the functionality. With the latest patch the inheritance tree looks
scene_t (mono)
|
--------------
| |
forkscene_t threadscene_t
scene_t (mono)
|
--------------------
| |
multiproc_t (abstract) threadscene_t
|
--------------------
| |
forkscene_t mpiscene_t
(Hopefully these diagrams work)
multiproc_t contains common functionality (the actually child worker
code), while the descendants override the communication functions (ie.
what is currently in ipc.cc will go into forkscene_t, MPI specific
stuff
will go into mpiscene_t.
Anyway, I'll work on the principle that the existing patch will go in
and produce a proof-of-concept of the MPI stuff and see how you feel
about including it.
Cheers,
Steve/Tarka
_______________________________________________
Yafray-devel mailing list
http://mail.nongnu.org/mailman/listinfo/yafray-devel

[Steve Smith]

Post by Diego Pino Navarro
Hi Steve,
Maybe my question is stupid(very),

Not at all.

Post by Diego Pino Navarro
but do you think there is a way to make yafray work under Apples
Xgrid cluster app, using your fork/thread modification + some coding?
I mean splitting an single render process in several independent
threads or forks that xgrid can effectively understand and that can
be send to every node in apples grid (and maybe as result get several
outputs(images) that can be batch joined together later?). I have just
tested xgrid using one/unique unix application over the network with
different parameters for each node, resulting in independent data
results that can be joined together later(math simulation), but it
looks like xgrid can make use of a threaded multi processor app and
distribute each job to a different node.

It sounds like it uses distributed-memory with process-migration, which
should work with the fork implementation. One other possibility is that
it requires the processes to be started independently with different
command-lines, is this the case. If so it would be more work, but
wouldn't be impossible.

Post by Diego Pino Navarro
I hope itÂŽs possible, anyway, i will try to compile your yafray with
your patch on my G4 and give it a test.

Definitely try it, it sounds like it could work. If not could you send
me any sort of developer info you have for how to develop for xgrid and
I'll see what it would take.

Cheers,
Steve