The Illinois eDream Institute is dedicated to promoting arts that are conceived, created, and conveyed through digital technologies
fiddler innovation fellowship

The Fiddler Innovation Fellowship is an annual student scholarship awarded by eDream to an undergraduate student who shows promise of significant, innovative achievements. The goal of the Fellowship is to inspire students to propose significant research projects that address cultural or global challenges that incorporate art and technology. With the support of eDream, the elected student will work with a faculty mentor or mentors of his/her choosing to develop and complete a substantial project. Funding for the Fellowship is based on student needs and the requirements of the proposed project.

Support provided by the Fiddler Innovation Endowment Fund.

Recent Fellows

Michael J Junokas – Ph.D. Candidate, Arts and Cultural Informatics

Michael J Junokas is a PhD Candidate in Arts and Cultural Informatics at the University of Illinois. He has a BA in Music Composition from Elmhurst College and an MM in Composition from DePaul University. His research is in developing innovative, multi-platform systems that have the ability to gather, interpret, process, and control signals in live artistic performance. Through the exploration of these systems, he hopes to create immersive technological environments artists can use for their own creative pursuits.

Mike is currently working with Dr. Robb Lindgren on ELASTIC3S (http://elastics.education.illinois.edu/), an NSF Cyberlearning research project conducted by an interdisciplinary team of learning scientists and computer scientists at the University of Illinois at Urbana-Champaign. The goal of this project is explore ways that body movement can be used to enhance learning of “big ideas” in science. For the project, he has focused on developing gesture-recognition algorithms trained using one-shot learning. His research from this project has been published in ICLS and NARST and has been used to inaugurate the Illinois Digital Ecologies and Learning Laboratory (http://education.illinois.edu/ideall).

Mike has also worked with Dr. Guy Garnett on the MovingStories project, which is an SSHRC-funded interdisciplinary, collaborative institutional research initiative for the design of digital tools for movement, meaning, and interaction. His research from this project has been published in  ACM C&C, ICMC, and ISEA.

Mike’s artistic and musical work (https://vimeo.com/junokas) has been exhibited at a variety of venues including McGill’s Transplanted Roots:Percussion Research Symposium, Illinois Wesleyan’s New Music Series, Illinois State’s New Sound Series, the School of the Art Institue’s Sullivan Galleries, and Experimental Sound Studio’s Outer Ear Series.

 

Kyungho Lee – Ph.D. Student, Illinois Informatics Institute

Kyungho Lee

Project: MovingStories
Mentor: Donna Cox
Proposal Abstract: The power of visualization arises from its capability to apply metaphors and semantics of graphical elements to help people perceive and understand new information in terms of their prior experiences and knowledge. Although various metaphorical visualization methods have become increasingly researched, only a few empirical studies exist which explore the matter of how people perceive and comprehend information represented by visual metaphors (visaphors) in interactive systems. The aim of my research is, therefore, to investigate how visaphors can convey the essence of information, and which aspects of visaphors can enhance the experience of the user using quantitative and qualitative methods. 

Kyungho Lee explores the potential of machine learning techniques to design intelligent interactive systems with an emphasis on the use of expressivity in body movement. He is a Ph.D. student at the Illinois Informatics Institute at the University of Illinois at Urbana-Champaign. He is a Fiddler Innovation Endowment Research Fellow and former Fulbright Scholarship recipient. 

Kyungho has been working with Dr. Donna Cox and Dr. Guy Garnett on the MovingStories project, which is an SSHRC-funded interdisciplinary, collaborative institutional research initiative for the design of digital tools for movement, meaning, and interaction. In the project, he used one of the XSEDE’s supercomputing facilities, Stampede (TACC), to analyze expressive conducting gestures to have better understandings of expressivity and experiential qualities of human body movement. His research outcomes have been published and exhibited in various venues such as IEEE VIS, ACM C&C, ICMC, ISEA and ACM SIGGRAPH Asia. 

Kyungho pursued a dual degree program at the Seoul National University in Korea, where he majored in Interaction Design and Information, Culture, and Technology Studies. Before he started a doctorate, he worked as an interaction designer with various clients for web services, mobile apps, and home appliances in Korea about six years.

Past Fellows

Colter Wehmeier – architecture

Colter Wehmeier

Currently: Researcher at the Cyprus Institute,/em>
Project:
RIVEEL3D
Mentor: Donna Cox
Proposal Abstract: As our lives become increasingly mediated by the internet, digital experiences take on an alarming amount of power in shaping reality. Like earlier mass media technologies, the internet frames our perception of the environment, other people, and ourselves. While the web radically decentralizes authority and democratizes access to information, the interplay of human psychology, the architecture of the internet, and its contingency on the real world, transforms the human condition in an unprecedented fashion. The essence of this change has less to do with the quantifiable aspects of computers than with the inherently complex and often unintuitive aspects of the people who use them.

RIVEEL3D is a digital archeology database embodied in a comprehensive 3d scanned model of Nicosia, Cyprus. I am responsible for user interface development in Unity 3d, and have experimented with VR headsets and physically tracked controllers to realize a museumquality educational experience. I’ve come to realize the limits of immersion/interaction in VR and have thus shifted my interests towards how we can use real physical space to map information in a humanintuitive way. Rather than build up an isolated virtual museum experience, I’m interested in letting smartphone users access data at real locations, based on what conceptual threads they follow. Full text of Proposal Abstract & Research Interests

In his time as an AVL SPIN Fellow, Colter worked on several major projects.

  • Designed and implemented GUI system, VR interaction model, and data recording/management for RIVEEL3D application (C# – Unity 3D)
  • Coordinated communications between NCSA and the Cyprus Institute
  • Devised software pipeline for translating photogrammetric and lidar data between archeology database and real time visualization engine (Unity 3D)
  • Documented and instructed researchers about RIVEEL3D software
  • Planned and directed student research office renovation project

RIVEEL 3D external links
NCSA Advanced Visualization Lab
Neapolis University of Paphos
The Cyprus Instititue

 

Austin Lin – theater

Austin Lin

Currently: Head of Technology at the White House
Project: Making Art Happen
Mentor: Donna Cox
Proposal Abstract: Today's technological world is probably best summed up by the now famous phrase: "there's an app for that". We now have so many applications (desktop, web, mobile, etc.) that end users are forced to use application after application to accomplish anything. Consider photography, we have applications for importing photos from your camera, for sharing photos, for editing photos, for printing photos, for creating movies from photos and so on. This situation is just as true in the "advanced" areas of computing like HPC and visualization. The next step in computing is not creating more systems but rather linking the systems we already have to create more powerful, more accessible and more intelligent systems. There is no magic bullet to this problem, and solving it is certainly beyond the scope of a one semester project or any one organization, but we can begin to create standards that move us towards more connected systems.

To this end, I propose to research and create recommendations for a control standard for interactive systems. The control standard would define how control end points are advertised, how messages are passed and how control devices or applications associate with the interactive system. In my researching I would focus on building upon existing standards when possible and work closely with my mentors at NCSA as well as others in the campus community who work with interactive systems. One example of an existing standard that I would utilize is the Open Sound Control (OSC) standard which is a message passing standard widely used in the performing arts world. OSC has an easy to understand address syntax, a large base of existing applications and a flexible specification making it an ideal starting place. I have also chosen to begin working with OSC because of my familiarity with it and the existing interest within NCSA's Advanced Visualization Lab (AVL).

The potential uses of such a control standard are far reaching, but a few that are specific to NCSA include: controlling AVL's vMaya from an iPad; feeding data from a performance venue to a simulation running on HPC resources at NCSA which in turn creates the graphics used in the performance; and synchronized showings of an interactive simulation in which control signals are fed into the simulation from multiple geographically distant locations. These are somewhat dry technical examples but the impact that the technology offers is profound. It means that the next Steven Hawking or Carl Sagan might be inspired by making galaxies collide using an iPad at Adler Planetarium. It means that using the same simulation, a dance choreographer might make galaxies collide using dancer's bodies and a Kinect, inspiring an entire audience. Suddenly a single simulation or application can do so much more and affect so many more people. This is what is possible when a control standard exists for interactive applications; it allows a community of people to build systems that can connect and in doing so become more than the sum of their parts. It democratizes the technology making it available to those who don't have the resources of NCSA.


 

NCSA Spin – Students Pushing Innovation

spin.jpg

2014-15 eDream SPIN Students

Sara Pelaez – New Media

Project scope: General projects
Mentor: Jeff Carpenter – Advanced Visualization Lab

Jasmine Shih – Computer Science

Project scope: General projects
Mentor: Donna Cox – Advanced Visualization Lab

Haoming Lai – Computer Engineering

Project scope: Advanced Visualization Laboratory general projects
Mentor: Donna Cox – Advanced Visualization Laboratory

Zhongshan Zeng

Project scope: General projects
Mentor: Donna Cox – Advanced Visualization Lab

Ziqiao Ding – Computer Science

Project scope: NCSA's Advanced Visualization Laboratory
Mentor: Donna Cox – Advanced Visualization Laboratory

Shuting Li – Electrical and Computer Engineering

Project scope: NCSA's Advanced Visualization Laboratory
Mentor: Donna Cox – Advanced Visualization Laboratory

2014-15 eDream SPIN Students

Hao Gao – Computer Science and Mathematics

Project scope: Autonomous, vision-based systems for monitoring construction projects
Mentor: Guy Garnett – NCSA Faculty Affiliate/eDream Institute

Hilal Habashi – Computer Science and Mathematics

Project scope: Investigate and develop mobile apps to enhance the experience of audiences of dance, theater, and music performances at the Krannert Center for Performing Arts
Mentor: John Toenjes – NCSA Faculty Affiliate/Department of Dance

Matthew Ho – Physics

Project scope: Developing gestural interfaces for high-level qualitative control of complex systems
Mentor: Guy Garnett – NCSA Faculty Affiliate/eDream Institute

Mingwei Hu – Computer Science

Project scope: Integrate AVL's Virtual Director camera choreography and remote collaboration software with yt, a fast parallel volume software renderer
Mentor: Donna Cox – NCSA Advanced Visualization Laboratory/eDream Institute

Annie Rong – Computer Science

Project scope: Integrate AVL's Virtual Director camera choreography and remote collaboration software with yt, a fast parallel volume software renderer
Mentor: Donna Cox – NCSA Advanced Visualization Laboratory/eDream Institute

Constantine Roros – Computer Engineering

Project scope: Developing gestural interfaces for high-level qualitative control of complex systems
Mentor: Guy Garnett – NCSA Faculty Affiliate/eDream Institute

Bentic Joseph Sebastian – Aerospace Engineering

Project scope: Developing gestural interfaces for high-level qualitative control of complex systems
Mentor: Guy Garnett – NCSA Faculty Affiliate/eDream Institute

Utsav Shah – Computer Science

Project scope: Investigate and develop mobile apps to enhance the experience of audiences of dance, theater, and music performances at the Krannert Center for Performing Arts
Mentor: John Toenjes – NCSA Faculty Affiliate/Department of Dance

2013-14 eDream SPIN Students

Shunhua Fu – Computer Science

Project scope: Developing a mobile port and GUIs for visualization and investigating the optimization of 4K compression and rendering
Mentor: Donna Cox – Advanced Visualization Laboratory

Shubham Gupta – Computer Engineering

Project scope: Working on integrating 3D gesture recognition into Virtual Director using Leap Motion
Mentor: Donna Cox – Advanced Visualization Laboratory

Yong Won Hong – Statistics

Project scope: Studying motion detection and the meaning of motion for use in human-computer interaction
Mentor: Guy Garnett – Illinois Informatics Institute

Colter Wehmeier – Architecture

Project scope: Developing new digital workspaces and presentation techniques by integrating methods from various virtual environments
Mentor: Donna Cox – Advanced Visualization Laboratory

Kiersten N. Jabusch – Illinois Informatics Institute

Project: Visualizing Moving Stories
Mentor: Guy Garnett – Illinois Informatics Institute/eDream

Jonathan Kirby – Computer Science

Project: Logging and Synchronization in Virtual Director
Mentor: Donna Cox – Advanced Visualization Laboratory

Austin Lin – Theater

Project: Making Art Happen
Mentor: Donna Cox – Advanced Visualization Laboratory

2012-13 eDream SPIN Students

Lauren Blackburn – Graphic Design

Project scope: Exploring visualization techniques for analyzing precision farming data and medical information
Mentor: Colleen Bushell – Visual Analytics

Jonathan Kirby – computer science

Project scope: Logging and Synchronization in Virtual Director
Mentor: Donna Cox

Zubin Pahuja

Laura Makdah

Nathan Russell – Industrial and Enterprise Systems Engineering

Project scope: Working on data modeling and machine learning techniques applied to precision farming
Mentor: Colleen Bushell – Visual Analytics

Sanny Lin – graphic design

Project scope: Visualizing Moving Stories
Mentor: Guy Garnett

Roshan Murthy

Shan Zou

 


Virtual Venues Festival

On April 30, 2014 as part of the SPIN student presentations, eDream student collaborators from CS493 Senior Project II will present a collaborative game played over the network with Univesity of California, Irvine. 

1. Overview

This Tele-Art Performance Event is a game played between audiences at each venue. The game is based upon the eight Effort Qualities of Laban Movement Analysis (LMA). Scoring is based on the audience’s ability to recognize the Laban Effort Qualities exhibited in the dancers’ movement improvisations.

The venues are joined via videoconference and a web application. Each venue projects the video conference image on one screen, the game ‘landscape’ on a second screen, and the scoring on a third. The game is played by audience members using a cell phone app. Dancers could either be only UCI dancers live at UCI and via teleconference at UIUC, or two sets of dancers live at each location. If UCI dancers only are performing, then scoring will have to be delayed according to the videoconference latency time amount.

Music is playing during the event. This music is tied  (somehow) to the progress of the game and/or movement improvisation. Alternatively, the music uses the Effort Qualities as the basis for improvisation, thereby giving more clues as to the Effort Shapes.

2. The Game

A) The event begins with a LMA expert delivering a short lecture/demonstration on Laban Effort Qualities.

B) Then the first level begins. A landscape (of some sort) appears on screen, as two dancers begin their movement improvisations, each one dancing with only one effort quality.

The audience must make an educated guess as to what effort quality each dancer is using. When they decide to guess, they press a button on the cell phone (or point their camera at an Augmented Reality (AR) target). This information is collected by the app and sent to the server for tallying. As they make their guesses, the landscape changes (somehow) to reflect this input. The scoring is also shown on the other screen, perhaps as Effort-Shapes images rising and falling in height as guesses are accumulated and tallied.

The first audience to reach a certain percentage of players guessing correctly wins that round (only X number of guesses per player). 

C) Additional Levels:

Level Two runs basically as Level One, only there are four dancers involved in the movement improvisation, and four Effort Qualities.

Level Three uses two dancers, but each dancer incorporates two Effort Qualities in their improvisation. The audience has to guess which two Effort Qualities are being used by each dancer.

Level Four uses four dancers, each dancer incorporates four Effort Qualities in their improvisation. The audience has to guess which four are being used.

More levels could be made, or another match could be played, starting at the beginning.

3. Discussion and Q&A

After the game is over, the winning side is the victor. There is a discussion and Q&A session between audiences at both venues.

This work is a continuation of the effort that started with the interactive application created for Mikel Rouse's "The Demo" which premiered in January at the Krannert Center for Performing Arts.

NCSA has a history of encouraging and nurturing innovative concepts. Some of the best ideas have come from highly motivated, creative undergraduate students working on NCSA challenges, or pursuing their own ideas with inspiration from colleagues and mentors. If you are looking for a unique opportunity to explore your potential, apply for a SPIN (Students Pushing Innovation) Fellowship.

Questions regarding the application process should be emailed to SPIN@illinois.edu.


eDream Summer Intern

Gabrielle Carels

Gabrielle Carels

Gabrielle Carels is a senior studying Digital Environments at the University of Michigan. She created this interdisciplinary major which explores the dynamics of new technology on media, culture and the intellectual process. With a curriculum comprising courses from the departments of English Language and Literature, Communication Studies, Screen Arts and Culture, American Culture, Computer Science, Informatics and the School of Information, her study in digital environments calls for a balance between building digital environments and analyzing the societal impact of these environments.

Gabrielle came to NCSA/eDream to immerse herself in a digitally intelligent environment, learn from top leaders in advanced digital visualization and further her education and knowledge of software tools. Gabrielle is currently working with Donna Cox and AJ Christensen to improve community and discussion on the XSEAD portal between scholars working across disciplines including design, the arts, engineering and science.


NCSA Spin – Students Pushing Innovation

spinstudents_1

SPIN Fellows to present on their work May 1

Questions? Contact spin@ncsa.illinois.edu.

Students participating in this semester’s SPIN Fellows program at NCSA will present a brief summary of their work. These 10-minute presentations will take place on May 1 from 3:30-5:30 p.m. in NCSA 1040. A pizza reception will follow. This event is open to the public.

  • “Visualizing Moving Stories” Kieri Jabusch, Junior in IPS: Human Computer Interaction and Sanny Lin, Senior in Graphic Design. Mentor: Guy Garnett
  • Kyungwha Byun, with title to come. Mentor: Kenton McHenry
  • “Making Art Happen” Austin Lin, Senior in Stage Management Program and Theater. Mentor: Donna Cox
  • “Logging and Synchronization in Virtual Director” Jonathan Kirby, Sophomore in Computer Science. Mentor: Stuart Levy
  • “Optical Music Recognition: Applications on Mobile” Robert Cheung, Junior in Computer Science and Finance. Mentors: Colleen Bushell and Michael Welge
  • “A Parallelized GDH-based Debugger” Nikoli Dryden, Junior in Computer Science. Mentor: Daniel LaPine
  • “Computer Science in Private Business” Michael Marshall, Sophomore in Math and Computer Science. Mentor: Merle Giles
  • “A Story about Twitter and Innovation” David Zmick, Freshman in Computer Science. Mentors: Colleen Bushell and Michael Welge

Documents:
SPIN proposal guidelines
SPIN proposal template

spin.ncsa.illinois.edu