All you need to know about Syphon & Isadora. Tutorial done by creator Mark Coniglio.
I recently stumbled across this amazing audio responsive software called Bazik.*
I have been after an audio visualisation tool for a while to incorporate into my Isadora VJ patches. And wow this one really hits the nail on the head.
I also love that “You can add Quartz animations or Shader GLSL” effects!
Here is a short demo on how I set it up.
*NOTE: this was limited to 100 downloads for the beta stage – very sorry!
“Z Vector is the world’s first professional visualization software based exclusively on modern depth sensors such as the Microsoft Kinect and the PrimeSense Carmine. It allows you to sample reality in real-time, visualize it the way you like and explore it in 3D with Full HD resolution. Create fresh material for your visual performances right there on the spot. Some great dancers in your crowd? Single them out from the background and make them part of your visual storyline. Assume control with our full MIDI/OSC support or mix Z Vector’s output with your existing set using Syphon (only on Mac OS X).”
I was lucky enough to meet Julius whilst I was in Helsinki for the LUX event and he gave me a sneak preview of it in action. I knew it was powerful then and its been 6 months (near enough to the day) now and wow it’s developed amazingly well.
I was lucky enough to get to test this amazing software well before the official beta test. I spent many hours playing around with parameters and creating some wonderful images. It’s been very hard to keep my mouth shut and even harder not to share any screenshots!!! I posted one or two that I knew I could get away with but nothing more.
What can I say? It’s much easier to head over to their website and either read about it there and/or download the beta whilst its active – go NOW!
What it is:
What it can do:
It was code named “The V” for a month or two but now it has a fancy new website and demo video from a band called Phantom and their official music video Scars.
Delicode are the driving force behind NI Mate and Kinect Stories. This third product is sure to blow the socks off any artist, VJ or individual who has the taste for live visuals and kinect style motion sensing trickery. With an already heavy hitting feature list I am sure this will be lots of fun!
I was lucky enough to see a demo of this when I met up with CEO of Delicode Julius Tuomisto in Helisnki for the LUX festival. Being able to single someone out in a crowd and then create visuas from them is awsome; podium dancers will be a brilliant addition to this software let me tell you!
I think people are going to love playing around with the editable GLSL shader editor; I seen it very briefly in january and it was a pop up window that looked a bit like processing.
I need to learn some GLSL coding I imagine. I also like the idea of Syphoning (is that even a word?) the results out into another software to add more layers, mix, map and play around with. Wow… can’t wait.
- real-time depth camera based 3D scanning and mixing
- multiple virtual camera and tracking modes
- multiple drawing modes (vectors, polygons, etc.)
- full sound and MIDI clock synchronization
- animatable GLSL shaders with editor
- full support for Windows / Mac OS X
- fully scalable output resolution
- stereo rendering (side-by-side, anaglyphic)
- GPU acceleration
- MIDI/OSC control
- Syphon support (Mac OS X)
This post is to help people understand and get started with Projection Mapping in Isadora. It is not an complete walk through or advanced post (you are best asking on the Isadora forum for advanced topics) but hopefully this will shed some light for new users, or those new to the idea of mapping.
What is mapping?
Mapping has been around for quite some time but my opinion is that it is the art of cutting up images and video to line up with complex shapes that are not the usual method for of display pictures and/or video. It allows artists to project onto buildings, cars, sculptures and custom built sets or similar. Basically anything that a light source (a projector) can hit!
Obviously you don’t have to use Isadora but one thing Isadora has that none of the other softwares has in the Interactive angle; motion tracking via Eyes and Eyes++ combined with the easy work flow and ever growing community.
Can Isadora do projection mapping already?
Isadora has had features that have allowed basic to fairly complex mapping for quite some time. It’s just the way in which you use them. Granted; it could be better but with a few 3rd part Quartz plugins 99% of things can be done easily.
Here are some basic actor that you should be taking a look at…
Isadora’s Existing Actors….
HINT: Hold SHIFT when clicking and dragging values for smaller increments and fine tuning!
- Great for projecting on very basic shapes.
- Helps you understand and learn about perspective and basic 3D perception.
- Ideal for beginners and not too complex.
- You can layer up multiple projectors
- Limited mapping functionality.
- Won’t do complex mapping (isn’t designed for mapping)
3D QUAD DISTORT
I use this one a lot. It’s really great for mapping and has enough features to explore new mapping skills but won’t overpower you with scary numbers or names.
HINT: Hold SHIFT when clicking and dragging values for smaller increments and fine tuning!
- You can pick each corner of the image/video and move it in an X and Y position.
- The Z translate (default -2.5) moves the image back and forth in 3D (3 dimensional) space.
- You can rotate the X, Y and Z axis.
- Has all the normal features of a the standard Projector actor.
- Not too overpowering. A nice step up from the Projector actor.
- You cant add corners or ‘break’ the image up into a grid.
- It can be tricky at first. Using Shift helps but depending on what you are mapping it can be a bit tedious.
- It can get complicated when you have a lot of these on your screen. You are best renaming the actor so you can see what’s what; such as “Top Right Window” or “DJ Booth – Left Side”.
Matthew Haber’s Mapping Tools.
Some of you knew this was coming! Pro user Mathew Haber has created a bunch of user actors that make mapping a lot easier. Based on the _1024 Quartz Composer
Matthew has created a topic just for support for the plugins on the Isadora Forum so please ask and discuss them HERE
The Cornerpin Mask user actor is in my eyes one of the best tools ever!
“This actors offers click-and-drag cornerpinning functionality for masking video. This has built in facilities for rectangular and oval masks and it also accepts a user Image input for any black and white image mask.”
- It’s just amazing! it works and makes life easier.
- The red circles that pop up when you turn Edit on allows you to click and drag the corner to where you want them. Not only this you can do it on the stage output… meaning you can map directly onto objects whilst looking at it. So yes, you literally move your mouse onto the output screen. NOTE: Check in isadora Stage preferences that your mouse is visible on stage output otherwise you won’t see it.
- Because this is a Mask; you can mask objects. Send in a black and white mask and it will block out areas you don’t want.
- Easily flick between the type of masks.
- it’s free – kind of…. Needs Core upgrade of Isadora.
- There are non really. Who can complain at this? Mathew has spent hours upon hours putting it together and creating it. If there was one negative it is simply that you need the Core Upgrade for Isadora and you have to instal a few bits and pieces; but once its done its done and you never need to do it again.
Syphon it out to somewhere else…
There are lots of other softwares that just do video mapping. And with apple you can send it out via Syphon to another software that can do it all with very advanced features. There is nothing wrong with doing this. Here are a few suggestions….
MadMapper. It’s been around for a while now and appears on all the main forums when you are talking about mapping. They where one of the first to create an external/stand alone mapping application and its developed along the way. It’s not free but it does a great job.
Visution Mapio is a powerful FreeFrameGL plugin for creating projection shows. Now you can work not only with the standard screen, but also with any inclines, including cylindrical and spherical and various shapes.
Shaped by Ixagon SHAPED is a user-friendly projection mapping software. It enables the user to easily map many different surfaces and attach content to them. The content is managed in a sophisticated sequencer similar to those found in video-editing and music-production software.
MeshWarpServer Quite a complex application. Ideal if you have a mesh/3D CAD designs of a building or object. A steep learning curve but it does have Syphon so you can send media in from isadora. Worth a look for those of you with time, patience and advanced 3D knowledge.
There are LOTS more; VVVV, Resolume Arena, Max, MSP, etc.
Hope this helps some of you.
Just a quick tutorial to get some of you use to colour tracking.
I have found colour tracking to be quite difficult to implement, this does not mean its impossible – just awkward at times!
Here is a screen shot of a basic colour tracking patch:
Using the chroma key you can filter the colour that is seen by the Eyes actor. Currently, the image above shows only the yellow colour being allowed through the actor. This means, only objects with the colour yellow will be tracked.
If you want to change the colour to be tracked, you can do this by changing the KEY HUE input, a bigger image of the Chroma actor can be seen below at its default state or ‘red’.
The output of the actor is then fed into the Eyes actor. You can adjust the threshold of the Eyes Actor to filter out any unwanted colour/noise and Eyes will work as normal (same with Eyes++)
A good idea is to change the key hue and then snapshot the colours using the snapshot features at the top of the Isadora window. Then you can jump between different colours much easier and the values will be stored. Then get creative!
EDIT: A very quick and very basic working example: