Triggering code from sound input: how to?

[dicky96]

Hi Guys

OK A quick bit about me so you know where I am coming from on this   :icon_cool:

I'm actually an electronics engineer from the 80s and 90s who worked on everything from TV and VCR to Computers and Industrial Electronics.  I did my City & Guilds in TV Video and Electronic servicing and my HNC (kinda like a University degree level in the old days when we had 'Polytechnic Colleges') in Microprocessor System Design and Programming - and I got distinctions throughout 

So I've got a good understanding of analog and digital electronics, and microprocessor systems.  I also have worked as a programmer, but as I am an electronics engineer at heart I tend to work 'backward' to most folks way of thinking, the higher the level of code the less I can figure out what the hell is going on! lol

So enough about me.

My current project is a frivolous one but something I have wanted to do for some years - so ready for next winter I am designing and building a computerised system to control outdoor xmas lights.  It's all for fun really

The questions I want to ask are about the general methods to create such as system, and what is the best approach... but read on and I'll tell you how far I got first

Yeah OK I know most folks who attempt this use professional DMX lighting controllers, but I'm an engineer so.....

A few years ago I got some old dis-used LED 'Moving Sign' displays - you know the ones that have the moving writing in shops and banks etc before we had flat screens everywhere.  I picked up three of these from a Radeio and Electronics fair for a good enough price.  I was gonna sell them on but never did anything with them as I couldnt find any info on how to interface them

Anyway what I have now done is dismantle one of these segns, as I figured it would contain lots of useful bits such as shift registers, which it does

I got three circuit boards full of shift registers, one nibble wide and 64 bits long - and then I reverse engineered the boards from looking up the components data sheets.

Having done that I set about interfacing one of the boards to an old PC (running XP) using the parallel port, I attached 64 strips of SMD RGB LEDs, I can control each strip with one nibble of data (actually I only use three bits pe LED strip as they have 8 colours - if we include off (black) as a colour

I connected it all up and it worked.

Writing some code based on Iczelions tutrials, an example called AsmIO, and the WinIO dll.  I can now run coloured pattern sequences, I have 64 LED strips each can be any of 8 colours... and it looks pretty cool  ;)  TBH as what I am doing is rather 'low level' ASM seems the best choice of programming languiage for this project yes?

I can also daisy chain more of these shift register boards and have even more lights

So thats what I've done so far

My program basically treats the LED display as a 'frame' of 64 nibbles, and loads a whole new 'frame' of data each step - so any of the 64 LEDs can be controlled on each step in any way I like

Now what I want to do is get the thing synchronised to the beat of music.  I don't want to 'step' the display once per beat, I want to 'step' it at a constant say 8 or 16 'frames' per second, and detect when a beat in the music occured so I can change the sequence direction or colours or whatever other parameter each time the music beats

I've done a bit of research and had a good think about this and can come up with three possible solutions:  What I would like is advice on these ideas, and maybe some ideas I haven't even considered

First off - I thought of getting my program working as a winamp visualiser.  From what I have read, visualisers are DLLs with a pretty much set structure, and are called by winamp after whatever mS delay you chose (say ever 80mS for example).  It looks like winamp sends your dll a 'message' like the windows messages in win32 api. It also looks like it gives you spectrum data and stuff.  I have an barebones example for a visualiser dll but it is written in pascal and I have never programmed in that language, but probably with a bit of help on datatypes and such from here I could get an asm visualiser running.  What I can't figure out is how to detect a beat in the music

Second off - I did a bit more looking and it seems I could create an 'APE' for the AVS visualiser.  APE is an AVS Preset Extension I belive. This is also a dll and I have a barebones example but that is written in C++.  If it was written in C I would have a go at creating an asm APE, but all that OOP stuff in C++ with classes and stuff is pure gobbledegook to me no matter how many times i have tried to learn it.  So I think I would need a lot more help with this.  On the plus side AVS calls your DLL at a framerate set from AVS control panel and also passes you a flag to tell you if a beat has occured since your program was last called

It did occur to me with method 1 or 2 above I could compile the barebones example rather than rewriting it in asm and then get that to call my dll but I really don't know exactly how to do this either

Thirdly - Well I am an electronics engineer.... I though maybe I should forget about all this winamp API, AVS and APE dll stuff.... and build a sound to light type circuit and connect it to my sound card output and to the input pins on the parallel port. Then my program could just read the data from the port.  I could add sensitivity control and a speed control to the external sound to light - and reading the clock pulse from the speed (oscillator) via the parallel port on one input and detecting a beat on another I would have external sequence speed and sensitivity controls.  While at it I could use op-amps/filters to detect beats (bass) and high end (snares/cymbals) and get my program to react to both.  At the moment my asm code to test the lights simply counts down a long loop before loading the next frame - probably not the way to do it as the CPU maxes out on my process at 99% and the sequence speeds up/slows down as the OS gives other processes time to execute.

So that's what I am up to.  I would really appreciate any suggestions or advice if this should all work and if I am approaching it the right way.


cheers
dicky

[dedndave]

it's funny you mention that
i had some ideas on programmable christmas lights
but, i was going a different way with it
seems like it might be rather profitable   

anyways, i wouldn't discard the WinAmp idea, at all
and - you can do similar drivers for Windows Media Player, as well

but - i would also think about having a direct-audio-in mechanism
one of the other forum members, Siekmanski, posted a method for using the sound ports
i can't seem to find it on the old forum, but he has played with this kind of stuff a lot   :P

you could do like an FFT thing, although that seems a bit over-kill-ish

[dicky96]

Hmmm  FFT also looks like a whole load of MATHS!

Maybe not my best forte

[dedndave]

not as bad as it looks
should be a breeze for an EE   
the concept is the important thing to understand...



http://www.commsonic.com/products/GenFftCoreDiagram.htm

[dicky96]

cheers for the replies dedndave

If I kinda copped out and went for the external sound detector plus clock generator, I figured it would not cost much to build - a couple of op amps to make high and lo pass filters and a couple of LED VU meter drivers and I could have a logic 1 input on one of the parallel port input pins for each 'bass' beat and also a logic 1 input on another pin for each 'treble' beat.  And add an external oscillator to control the speed of my sequencer by putting clock pulses on a third pin of the parallel port

With a bit more logic circuitry to latch the 'beat' signals I could also have the facility to 'clear' the current beat input once my code had dealt with it


Now if I did all that, how would I tell my code to load the next 'frame' to my LED display once on every clock of the external oscillator (when it can also check if a beat has occured)

I mean without having my program running a continous loop polling the parallel port inputs to see if something happened (I'll use three bits on the port control register to detect clock / low frequeny beat occurred / high frequency beat occured)

So I need some way to put my process to sleep once it outputs a frame of data, then have it triggered when there is a clock pulse on the parallel port to output the next frame.  How would I go about doing that?

I reckon if that's possible, then it's probably the easiest way to do this.

If anyone is interested in all this hardware/asm combination then I'll post up my circuit diagrams of course

Cheers
dicky

[dedndave]

you can do it that way
but - hardware interrupts and such are easier in 16-bit code than 32-bit

personally, i would try to implement it in software
and - i would probably use the serial port - actually, a USB dongle programmed as a serial device
minimize hardware construction costs - maximize profits   

although, i can't say i have any experience with the parallel port in 32-bit, yet
i can't see how you'd detect a hardware transition like we used to in 16-bit days   :P
one of the other forum members may be more helpful

it was easy in DOS - you could revector an interrupt and away you go

[dicky96]

Hmmm OK  thanks for the continued input on tis amd I hear what you are saying. 

And its not about profits in this case - I'm doing this purely for my own enjoyment

And I did say from he start as an Engineer I tend to think of things 'backwards' compared to modern software thinking so now you see what I mean

In as much as my mind tends to reduce things to the hardware 'whats going on here' sort of level

So as I am gonna be using this PC (lets face it we all have lots of old PC hardware lyeing around) very much so to control these lights - so maybe I just boot it from an old DOS disc and do it that way..   But then it would be nice to have the audio source coming from the same PC (like in winamp)

Another approach on this then - How about I get windows OS to give my process a kick every 62.5mS (16 times a second) or whatever time I set, rather than an external clock causing an interrupt? 

Then check for a high or low 'beat' on the parallel port and step my LED sequence accordingly?

Is that reasonably do-able??   It seems like a compromise between your way of approaching this and mine  :icon_cool:  So how then do I get windows to kick my process on a regularly timed basis?


dicky

[jj2007]

How about I get windows OS to give my process a kick every 62.5mS (16 times a second)

That's easy: SetTimer plus a callback proc, or if you go for a non-GUI app:
.While 1
  invoke Sleep, 62
  ... do stuff ..
.Endw

[dedndave]

you could use a machine with win98 very easily
that was the last decent windows OS that let you access I/O ports, directly

you can use SetTimer to generate a message
windows will send you a WM_TIMER message at the interval you specify
the smallest time increment is 10 mS - so that would be great

still, the sound is not going to be syncronized with the timer   :P

as i said, i have not played with the parallel port under win32, yet
i am sure someone in here has, though   :P

btw, i am an engineer, as well
i have been tinkering with electronics since age 6 - i am now 57
when i was a kid - solid state was "that new stuff" - lol

[MichaelW]

You may be able to do some good with a driver that supports direct I/O port access, like the one available here:

http://www.internals.com/

Somewhere on the old forum I posted import libraries and include files for the previous version, and I think also for the current version (3.0).

And FWIW, Windows 98 First Edition would allow 16-bit DOS apps very free access to the hardware. I once did a series of 16-bit DOS apps that could run an ATA Identify Drive command on the hard drive through the command/control block registers. The other versions of Windows 9x all blocked access to the command/control block registers, but under 98 FE it worked apparently just as it did under DOS. I recall thinking that it might be possible to set up some sort of reasonably efficient communication between a 16-bit DOS app and a Win32 app, with the DOS app interfacing with the hardware and the Windows app interfacing with the user, but I don't recall testing this. And you can intercept hardware interrupts by placing a handler directly in your DOS app. 

[dedndave]

well - micro$oft kind of put a wrench in all that stuff, seeing as they require drivers to be signed under vista or newer
pretty shitty of them, if you ask me
there are work-arounds for that

my guess is - the kernel mode stuff to access parallel and serial ports already exists in the OS
we just have to figure out how to use it   :P
that is going to be the best solution, because we don't want to pay m$ money everytime we want to access hardware

we need to figure out how to talk to the ports, directly
we need to get the thing to send us a message when a hardware interrupt event occurs
namely - the hand-shaking lines on parallel and serial ports

i used to use these lines to detect hardware conditions all the time   :(

[Donkey]

Hi dicky96,

DirectShow offers a filter for audio capture.

http://msdn.microsoft.com/en-us/library/windows/desktop/dd317585%28v=vs.85%29.aspx

If you'd like to avoid DirectShow there is the waveIn API

http://msdn.microsoft.com/en-us/library/aa446573.aspx

[dicky96]

Cheers guys that give me plenty of options to explore  :t

Yeah I could put win98 on the PC as it is only to be used to control this lighting system, however I am currently running XP on it and yes I am already using that winIO dll from internals.com that someone mentioned, to access the parallel port. 

It's working sweet, I can output data through the port no problem so thats running fine

Just out of interest the way I send data to the shift register board at the moment, is to present a nibble on the lower bits of the data register at 0378h.  Then I raise the strobe line high and then low again (bit 0 on the control register port 037ah) to clock the data into the shift register

I repeat that 64 times to clock in the whole 'frame' of data, it shifts one location down the shift registers each time I clock.  Once all the 64 nibbles of data is loaded (doesn't take very long at all) I use the LineFeed (bit 1 of control register) to enable the outputs on all the shift registers which 'displays' the data on the LED strips

I then process the frame of data to create the next step in the light sequence, and wait some milliseconds before laoding the next frame

Here is my nasty bit of code that I used to test all the hardware was working OK

Code Select Expand


;------------------------------------------------------------
; Win32Asm stupid sample for WinIo v1.2
;------------------------------------------------------------
;
.486
.model flat, stdcall
option casemap:none

include \masm32\include\kernel32.inc
include \masm32\include\user32.inc
include \masm32\include\windows.inc
includelib \masm32\lib\kernel32.lib
includelib \masm32\lib\user32.lib

includelib WinIo.lib ;<<<<<<<<<<<<<<<<<<<<<< Library
include WinIo.inc ;<<<<<<<<<<<<<<<<<<<<<< Include file

.data
frames dd 01h ; number of frames
framesize dd 40h ;(frame size = 64 decimal
delayvalue dd 00fffffh
clockwidth dd 0ffh

;frame0 db 01h, 02h, 04h, 08h, 10h, 20h, 40h, 80h,
; 80h, 40h, 20h, 10h, 08h, 04h, 02h, 01h,
; 01h, 03h, 07h, 0fh, 1fh, 3fh, 7fh, 0ffh,
; 0ffh, 7fh, 3fh, 1fh, 0fh, 07h, 03h, 01h
;frame0a db 01h, 02h, 04h, 08h, 10h, 20h, 40h, 80h,
; 80h, 40h, 20h, 10h, 08h, 04h, 02h, 01h,
; 01h, 03h, 07h, 0fh, 1fh, 3fh, 7fh, 0ffh,
; 0ffh, 7fh, 3fh, 1fh, 0fh, 07h, 03h, 01h 
frame0 db 01h, 00h, 00h, 00h, 00h, 00h, 00h, 00h,
02h, 00h, 00h, 00h, 00h, 00h, 00h, 00h,
03h, 00h, 00h, 00h, 00h, 00h, 00h, 00h,
04h, 00h, 00h, 00h, 00h, 00h, 00h, 00h
frame0a db 05h, 00h, 00h, 00h, 00h, 00h, 00h, 00h,
06h, 00h, 00h, 00h, 00h, 00h, 00h, 00h,
07h, 00h, 00h, 00h, 00h, 00h, 00h, 00h,
00h, 00h, 00h, 00h, 00h, 00h, 00h, 00h 
 
.data?


.code
start:
invoke InitializeWinIo

; control register is inverted.
;bit0 = clk (0 is hi 1 is low)
;bit1 - strobe (0 is hi 1 is low)
mov eax, 03h
invoke SetPortVal, 37Ah, eax, 1 ;set port control register: clk low strobe low

; FRAME OUTPUT
;-----------------------------------------------------------
outputframe:
lea edx, frame0 ;start of data
mov ecx, 00h

.while ecx < [framesize]
mov eax, 00h
mov al, byte ptr [edx+ecx] ;get data byte

push edx
push ecx

invoke SetPortVal, 378h, eax, 1 ;output data
mov eax, 02h
invoke SetPortVal, 37Ah, eax, 1 ;strobe low clock high
call clkwidth
mov eax, 03h
invoke SetPortVal, 37Ah, eax, 1 ;strobe low clock low

call delay
pop ecx
pop edx

inc ecx
.endw

mov eax, 01h
invoke SetPortVal, 37Ah, eax, 1 ;strobe high clock low to display frame
; END OF FRAME OUTPUT
;----------------------------------------------------------------------

;Build Next Data Frame
lea edx, frame0
mov ecx, 00h
mov ah, byte ptr [edx + 3fh] ; get last byte of frame

.while ecx < [framesize] 
mov al, byte ptr [edx + ecx]
mov byte ptr [edx + ecx], ah
shl eax,8h
inc ecx
.endw
jmp outputframe

invoke ShutdownWinIo
invoke ExitProcess, NULL

delay: mov ecx,[delayvalue]
; delay loop
.while ecx >0
sub ecx,1
.endw
ret

clkwidth: mov ecx,[clockwidth]
; delay loop
.while ecx >0
sub ecx,1
.endw
ret


end start


That runs a nice sequence of coloured LED bars

PS @dedndave I',m just a few years younger at 53 mate and like you started to mess around with electronics at a young age so it seems we have plenty in common  :icon_cool:.  Even though I took my C & G in TV repair in 1983 we still had to learn Thermoinic valve circuitry and I certainly worked on valve sets back in the day, there were plenty still kicking around well into the 80s!

All the help and suggestions much appreciated

dicky

[Siekmanski]

Hi dicky96

Very fast beat detection can be done with the Goertzel algorithm to get 1 frequency from the audio spectrum.
You only have to detect 200 Hz to get the bass beat.

For more band detection you can use FIR filters for each frequency band you desire.

At last FFT if you need the whole audio spectrum.

What about using FTDIChips to communicate via USB with your hardware?

I use the FT232RL often to communicate with microcontrollers.
You can bitbang in high speed.
Data transfer rates from 300 baud to 3 Mbaud (RS422, RS485, RS232 ) at TTL levels.
The FTDI chip has an internal timer and fifo buffers, so no need for a windows timer, just set the transfer rate you need.

This is why I did the FTDI-Api translation to Masm ( my favorite language )

Be sure you have installed the Direct (D2XX) drivers and not use the Virtual COM Port (VCP) drivers.
The D2XX driver allows direct access to a USB device via a DLL interface.

link to FTDI home page:                     http://www.ftdichip.com/index.html
link to programmers guide:                http://www.ftdichip.com/Support/Documents/ProgramGuides.htm
link to D2XX_Programmer's_Guide:    http://www.ftdichip.com/Support/Documents/ProgramGuides/D2XX_Programmer's_Guide(FT_000071).pdf
link to application notes:                    http://www.ftdichip.com/Support/Documents/AppNotes.htm
link to BitBangMode FT232R-FT245R: http://www.ftdichip.com/Support/Documents/AppNotes/AN_232R-01_Bit_Bang_Mode_Available_For_FT232R_and_Ft245R.pdf
link to ChipID FT232R-FT245R:           http://www.ftdichip.com/Support/Documents/AppNotes/AN232R-02_FT232RChipID.pdf

[Siekmanski]

Another approach is to connect your audio output to an adc input from your microcontroller
and then use a Goertzel algorithm to detect the tones you need.