Parag K Mital

tensorflow_tutorials. From the basics to slightly more interesting applications of Tensorflow

CADL. ARCHIVED: Contains historical course materials/Homework materials for the FREE MOOC course on "Creative Applications of Deep Learning w/ Tensorflow" #CADL

pycadl. Python package with source code from the course "Creative Applications of Deep Learning w/ TensorFlow"

time-domain-neural-audio-style-transfer. NIPS2017 "Time Domain Neural Audio Style Transfer" code repository

dance2dance. Discrete Figures AI: Seq2Seq of Dance Data (Mikiko + Daito Manabe + Kyle McDonald)

pkmFace. facial shape modeling, appearance modeling, and head pose recognition. Uses Jason Mora Saragih's FaceTracker code to track facial landmarks; GreatYao's aam-library for building/reprojecting the model (which may in fact be an uncited port of Jason's DeMoLib).

Real-Time-Object-Detection. real time object detection using opencv and simple UI to select the object to track

flask-uwsgi-tensorflow. Example of serving a TensorFlow model with Flask and UWSGI

KinectHandAnalysis. Does finger tracking, contour analysis, and different types of shape description for the hand contour using a 3D ROI around the hand-tracked by NITE middleware, using OpenCV 2.2, an OpenGL scene, and OpenNI.

pkmFFT. pkmFFT provide simple interfaces to the Accelerate.framework for performing vectorized FFT. pkmSTFT builds on pkmFFT to perform Short Time Fourier Transform efficiently using vectorized ops. Also handles options for windowing. pkmDCT provides a simple discrete cosine transform using Accelerate and pkmMatrix.

CycleGAN. Describing CycleGAN (excerpt from CADL course on Kadenze, Inc.)

ofxCaffe. openFrameworks addon for visualizing and interfacing with pre-trained models in Caffe: Convolutional Architectures for Fast Feature Embedding. Requires Caffe, openFrameworks 64-bit, glog, hdf5, OpenCV, CUDA, pkmMatrix, and pkmHeatmap. Pre-trained models not included but can be found linked in Caffe's "Model Zoo" and placed in the bin/data directory of the example project.

MTEC-498-698A-01-Algorithmic-Sound. Generative Sound Practice w/ OpenFrameworks

MTIID-MTEC-616-01-2017. Materials for "Audiovisual Interaction w/ Machine Learning", Bachelors (416) and Graduate (616) course @ CalArts MTIID MTEC

pkmOpticalFlow. Interface for OpenCV's TV-L1 Duality Based Approach to Optical Flow. Does some additional tricks to speed up processing, allow for analysis of the spatio-temporal changes in flow, and visualize the results.

autoEncoderSynthesis. Performs interactive audio synthesis using a previously trained auto encoder. VIsualizes hidden layer and allows for interaction with hidden units. W/ Andy Sarroff. See Andy's github (woodshop) for more interesting projects using Deep Learning and audio synthesis.

pkmAudio. Sound synthesis library. Implements a number of feature databses including frame-based, segment-based, and sequence-based audio analysis/storing/retrieval. Also includes libraries for GPS-based synthesis including binarualization retrieval. Complements many other libraries including pkmBinaural and Memory Mosaic. Heavy dependence on vectorized ops using pkmMatrix and OSX's library Accelerate.

StickFigureOSC. Streams PrimeSense NITE's Skeleton Data via OSC (XCode Project)

CARPE. CARPE is "Computational Algorithmic Representation and Processing of Eye-movements". It supports visualizations of film/eye-movements in columnar format, peak-through visualizations, heatmaps, optical flow analysis, gaze clustering, and many other visualization options. See http://thediemproject.wordpress.com for more details. Compuational and Algorithmic Representation and Processing of Eye-movements - OSX Version

phdThesis. Ph.D. Thesis from Goldsmiths, University of London entitled, "Audiovisual Scene Synthesis". Hosts all images and latex files.

pkmSIFTImage. Creates a dense SIFT image description and displays the image based on a PCA reprojection. Based on SIFT Flow code by Ce Lui, Jenny Yuen, and Antonio Torralba.

pkmBlobTracker. Background modeling for foreground subtraction, tracks multiple blobs (people), their orientations (using leading motion vector), and has a nice visual display for seeing the results... video demonstration here: http://vimeo.com/22054133 - more info here: http://pkmital.com

pkmColorBlobTracker. track overhead using color and map tracked points to a new geometry using a homography transformation and calibration routine - some example test videos are provided in the bin/data directory of an overhead capture. The tracking transformation is useful for when you need a defined metric space of your tracking parameters, or need to account for different user heights in tracking their paths in a space.

NSCARPE. CARPE is "Computational Algorithmic Representation and Processing of Eye-movements". It supports visualizations of film/eye-movements in columnar format, peak-through visualizations, heatmaps, optical flow analysis, gaze clustering, and many other visualization options. CARPE using NSWindow, Multiple Heatmaps, XML file settings, Difference of Heatmaps, GPU optimizations

pkmDetector. 3d Object Tracking and Pose Estimation for the iPhone

CEMA. openFrameworks projects from my course at Srishti's School of Art, Design and Technology's Center for Experimental Media: http://pkmital.com/home/teaching/cema-workshop/

pkmProjectionMapper. openFrameworks based projection mapping for distorting or mapping a collection of drawing commands/videos/images etc... needed a lightweight and robust projection mapping utility though lpmt was too heavy. homography code stolen from lpmt.

pkmPhaseVocoder. Phase vocoder using pkmFFT and pkmMatrix. Does not seem to do as well as commerical applications. Would love it if anyone made use of this and improved upon it.

ofxiTunesLibraryStream. simple way to stream 32-bit float data from itunes in real-time

seq2seq. Exploring audio modeling / synthesis w/ seq2seq

pkmKalman. Kalman filter optimized for iOS/OSX

BregmanToolkit. Audio and Music Analysis and Synthesis in Python

eegLogger. For streaming/logging emotiv eeg headset data

videoSPAM. [in progress] synthesis of audiovisual material from youtube!

pkmMatrix. pkmMatrix provides a lightweight Matrix class using the Accelerate.framework for vectorized operations

pkmBinaural. For performing GPS based concatenative sound synthesis, ANN retrieval based on GPS locations, HRTF based binauralization (mono->stereo using HRTF FFT-based Overlap-Add Convolution) using the IRCAM HRTF Database

ofxOpenCV2461. ofxOpenCV linking against OpenCV 2.4.6.1, including libraries for OSX

caffe-tensorflow. Caffe models in TensorFlow

eegShapeAnimations. testing some encoding/decoding using EEG. this produces various animations that are co-registered with the recording of eeg.

tensorflow-python3-jupyter. Docker container with python 3 version of tensorflow accompanied by jupyter

ai-audio-startups. Community list of startups working with AI in audio and music technology

eegGaborAnimations. testing eeg encoding/decoding. this produces some gabor wavelets and co-registers the visualization and recording of eeg.

pkmDPGMM. Interfacing libcluster for doing Variational Dirichlet Process Gaussian Mixture Models. Depends on Eigen3 and pkmMatrix. Libcluster included.

SendTextPlus. An improved SendText plugin for Sublime Text. Send text to various programs. Works with R, Python and Julia

magenta. Magenta: Music and Art Generation with Machine Intelligence

CADLBook. Testing

tensorflow. Computation using data flow graphs for scalable machine learning

UnrealOpenAIPlugin. C++

freesoundDownloader. Uses cURL to download from Freesound.org

pkmEXTAudioFile. pkmEXTAudioFileReader and pkmEXTAudioFileWriter provides simple interfaces to reading and writing audio files.

pkmRecorder. For streaming/recording audio files, circular buffers (see pkmMatrix and pkmEXTAudioFile as well)

pkmPoseCalibrator. Calibrate head pose with respect to a screen (television/monitor) for an attention-based measure. Uses Jason Mora Saragih's FaceTracker, please contact him for the code.

lumin-order. LuminOrder: An OS X app to reorder a video in order of its frames' brightness levels.