Duma Server side

Duma

Server side

• Secretary – central core of the system, dispatcher for all jobs and interaction manager w/ users. You don't need to keep running dispatchers on each user’s workstation, “Fire and forget” :)

• Comrade – remote launching service

Client side

• Deputat – main user GUI for management all jobs and users

• Televisor – tool for getting rendered images over Internet

• Scripts – ready to use way schedule render from native application interface: Renderman (MTOR/PRS), MR standalone (MayaToMR), MayaBatch MR, MayaBatch Software, Nuke, Shake, DF, After Effect and others

• DumaStart – tool for fast and easy creation custom Alfred Scripts.

• Rrun – console client for manual launching command on remote Comrades. Useful in testing purposes.

Compatibility with Alfred®

• Duma has full Alfred’s® functionality, so you shouldn't change your render routine. And of course we offer many new abilities and fresh, user friendly GUI :)

• Alfred Scripts for render job describing through full size TCL pre-processor

• 100 % compatibility with all MTOR/PRS generated Alfred Scripts

• Support distributed and deferred Mtor (or Houdini) Rib generation with dynamic creation of subtasks branches

• Alfred® and RenderMan® are registered trademarks of Pixar Animation Studios

Stability and speed

• About 2 year's of VFX production on our humble ~100 CPUs render farm

• Wide hardware’s platform support Linux, Mac, Win

• Duma central module – Secretary, takes advantages of modern multi core CPU for it’s scalability. Also Secretary uses proprietary database for interactive work w/ thousands Comrad's, many Deputat's, Gigs of output logs

• Tested for manage large render farms. Tests was conducted on render farm which counted about 1500 virtual hosts

Users and jobs

• In Deputat you may select several render queues to merge theirs jobs for collectively manage (see video demo)

• For jobs execution Duma use some kind of Alfred®’s job spillover scheme

• Every Queue/Job has accumulating Priority property. Queues/Jobs with high priority can consume more CPU’s than other

• Maximum running task limit may be applied. Separate limit values for queue and job may be set

• Inside queue Job Blocking Fence may be set. Jobs beyond fence will not ran while all jobs before become completed. E.g. Job 1 – textures gen, Job 2 – render which uses this textures

Security

• Users – can manage only their private queue and any public queues. Also any people can overview any other queues/jobs/tasks

• Supervisors – can do what they decide to do. E.g. start/stop and even delete any jobs, change any parameters. Also that’s people can boost Priority factors in values greater than 1 (one)

• Any activities on private queue from any person except queue owner (i.e. supervisor) are logged on Secretary side. So any people can investigate what happened with their render tasks

• For easy login, (w/o credentials) on office sites trusted IP networks can be specified. User which connecting from other than trusted sites must use name/password to login

• User accounts can be pooled from domain or local computer database

Output and system Log's

• Live delivery task’s stdout/err. Instant browsing regardless of total output’s size

• Out log for each run session store separated

• Auxiliary log detail command execution history (see it in video presentation)

• Useful std out application/options:

• Adjustable RegExp for live task completion’s percent calculation
• Output filters (w/ counters) for sweep out hordes of duplicated warning/errors
• Syntax highlighting (errors, warning and etc.) may be set for particular task’s classes

Working from anywhere

• Lightweight network protocol to Deputat allow perfect working over Internet. Most likely you will not feel any deference while being working from home

• Deputat sends requests describing desired to view area, while Secretary accumulated that incremental requests to know Deputat’s interests. When something in desired area has changed, Secretary send corresponding data bit to reflect changes

• Secure login - user connecting from Internet must provide name/password credential to login

• Televisor – delivering pictures from your local net to your home. By your request, on server side, images will converted to jpeg with desired size and quality, and then immediately sent to you directly through TCP (see video demo)

Farm Stats

• Runing task - list containing all running task on farm. Multi selection to group management of running task is available.
• Hosts - list w/ attached to Duma computers. Live host CPU’s loading, free memory and other stats available from here. Also, missing hosts highlighted to red colour, while hosts excluded from render schedule marked with cross stroke
• Selection in one of that’s tables lead to auto-selection corresponding rows in another. E.g. selecting running task you also going to select host on which that task executing

Schedule reliable render in few clicks!

• Send render job to Duma directly from your favourite application using one of our ports

• Auto testing render output files. Analysis performed by easy to customize script. At this moment output file’s modify time and size similarity are checked.

Schedule file for Secretary

• Single XML file describing various aspects of rendering policy may be modified and then automatically reloaded by Secretary on the fly

• “Rules” – much more flexible than classical “slot” system. Using TCL you may write own strategy how use render frame. E.g. “slot system” may be described by single instruction return [d_ServiceCounter *]<1.0

• Job/Computers Variables – may be used in Rule Expressions and easy modified from Deputat GUI. E.g. you may declare variable “memory_demand”, then you should reflect secretary’s rule expression by adding compare that variable and host’s memory. After that/ you may easy sift out low memory hosts for particular job

Statistic

• After render task has completed, info describing that render event send to external SQL database (e.g. PostgreSQL). In particular that info contain:

• Project/Scene/Shot
• Start Time + duration
• Owner name, task type, render host, exit code and many others…

• You may collect render statistic from that SQL database and format it into nice report using any suitable software

• Custom report solution may be developed for you. Basically we provide source codes (c++) of SQL logging utility which can be modified to reflect your specific demands

Administration

• Secretary and Comrade:

• Simple error statistic provided internally (w/o any SQL), give easy way to discover buggy and unstable render host
• XML config files give very fine control over Secretary amd Comrade
• All auxiliary and trace info collected in batch of log files. You may setup size limit and time of live that's logs

• Deputat config's:

• User Config file. Store all GUI element settings, as well as many other parameters and special config file for new user
• globals override for all site
• Auxiliary configs purposed to define set of viewer’s tools and setup highlighting of output log

Materials and contacts

• Video demo (on-line) Video demo (zip 118mb)

• Power Point Open Office

• Duma configs samples

• Ivan Ivanchik [e-mail]

• Konstantin Kharitonov [e-mail]

• © 2008,Cinesoft