edit

ReSpeaker 4-Mic Linear Array Kit for Raspberry Pi

enter image description here

Seeed’s ReSpeaker 4-Mic Linear Array Kit is an extension board, aka HAT designed for Raspberry Pi. It's a linear microphone array kit, comes with four microphones and designed for AI and voice applications. That means you can build a more powerful and flexible voice product with Raspberry Pi which can integrate Amazon Alexa Voice Service, Google Assistant, and so on.

ReSpeaker 4-Mic Linear Array Kit for Raspberry Pi consists of two boards, one is voice accessory HAT, another is four microphones linear array.

ReSpeaker 4-Mic Linear Array Kit for Raspberry Pi support 8 input & 8 output channels in Raspbian system. The first 6 input channel for microphone recording(only first 4 input channels are valid capture data), rest of 2 input channel are echo channel of playback. The first 2 output channel for playing, rest of 6 output channel are dummy.

Features

  • 2 ADC chips and 1 DAC chip
  • 8 input and 8 output channels
  • Four microphones array
  • Grove support
  • Compatible with Raspberry Pi 40-pin headers
  • Headset and speaker voice output

Specification

  • 2 x X-Power AC108 ADC
  • 4 x high performance analog microphones
  • 1 x X-Power AC101 DAC
  • Voice output:
    • 3.5mm headset audio jack
    • Speaker jack
  • Compatible with Raspberry Pi 40-pin headers
  • Microphones: Knowles SPU0414HR5HSB
  • Sensitivity: -22 dBFS (Omnidirectional)
  • SNR: 59 dB

Application Ideas

  • Smart speaker
  • Intelligent voice assistant systems
  • Voice recorders
  • Voice conferencing system
  • Meeting communicating equipment
  • Voice interacting robot
  • Car voice assistant
  • Other scenarios need voice command

Hardware Overview

Assembly drawing

Getting Started

Hardware

Prerequisites

ReSpeaker 4-Mic Linear Array Kit x1

Raspberry Pi 3B or 3B+ x1

Micro-USB Cable x1

PC x1

Earphone or Speaker x1

Tips

Actually the ReSpeaker 4-Mic Circular Array support Raspberry Pi Zero, Raspberry Pi 1 B+, Raspberry Pi 2 B, Raspberry Pi 3 B and Raspberry Pi 3 model B+, in this wiki we are using Raspberry Pi 3.

Connection

Step 1. Connect the ReSpeaker Voice Accessory HAT with ReSpeaker 4-Mic linear Array via the Ribbon Cable

Step 2. Plug the ReSpeaker Voice Accessory HAT into the Raspberry Pi via the 40 Pin GPIO

Step 3. Plug the earphone into the 3.5mm headset audio jack or plug the speaker into the JST 2.0 speaker jack

Step 4. Connect the Raspberry Pi with the PC via the micro-USB cable

Pics here

Software

Prerequisites

Plan A

PUTTY

You need to use Putty or other ssh tools to connect with your raspberry pi. Before started, please make sure:

1- Open ssh fuction of your pi to let the putty in. If you do not know how to open ssh, please google how to setup ssh raspberry pi

2- Your raspberry pi and your PC are working on the same WiFi net. If you do not know how to config WiFi, please google how to setup wifi raspberry pi

3- Get the ip address of your raspberry pi, if you do not know how to get the ip address please refer to raspberry offical documentation

4- Use the ip address to connect the raspberry pi with your PC via putty ssh serve.

pic

Then please tap the host name and the password. the default ID is pi and the password is raspberry.

login as: pi
pi@192.168.43.210's password:raspberry

Now you are in, and you can tap the command in putty and play with your raspberry.

VNC Viewer

To make this kit work with alexa or dueros, you need to open a web site to get the authorization. So you need to use VNC Viewer to log in your amazon or baidu account. So please make sure you have open the VNC service of your raspberry.

Or you can just consider plan B.

Plan B

If you are tired of all above, you can just use a HDMI Monitor and plug the USB Keyboard and USB mouse in to your raspberry, it works too, simple and easy.

Step 1. Install seeed-voicecard

Get the seeed voice card source code. and install all linux kernel drivers.

sudo apt-get update
sudo apt-get upgrade
git clone https://github.com/respeaker/seeed-voicecard.git
cd seeed-voicecard
sudo ./install.sh 
sudo reboot

Step 2. Check the Sound Card

Tap the command below to check the record device.

pi@raspberrypi:~ $ arecord -L

It should be like:

null
    Discard all samples (playback) or generate zero samples (capture)
default
playback
dmixed
ac108
multiapps
ac101
sysdefault:CARD=seeed8micvoicec
    seeed-8mic-voicecard, 
    Default Audio Device
dmix:CARD=seeed8micvoicec,DEV=0
    seeed-8mic-voicecard, 
    Direct sample mixing device
dsnoop:CARD=seeed8micvoicec,DEV=0
    seeed-8mic-voicecard, 
    Direct sample snooping device
hw:CARD=seeed8micvoicec,DEV=0
    seeed-8mic-voicecard, 
    Direct hardware device without any conversions
plughw:CARD=seeed8micvoicec,DEV=0
    seeed-8mic-voicecard, 
    Hardware device with all software conversions

Use the following command to check the play device.

pi@raspberrypi:~ $ aplay -L

It should be like:

null
    Discard all samples (playback) or generate zero samples (capture)
default
playback
dmixed
ac108
multiapps
ac101
sysdefault:CARD=ALSA
    bcm2835 ALSA, bcm2835 ALSA
    Default Audio Device
dmix:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Direct sample mixing device
dmix:CARD=ALSA,DEV=1
    bcm2835 ALSA, bcm2835 IEC958/HDMI
    Direct sample mixing device
dsnoop:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Direct sample snooping device
dsnoop:CARD=ALSA,DEV=1
    bcm2835 ALSA, bcm2835 IEC958/HDMI
    Direct sample snooping device
hw:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Direct hardware device without any conversions
hw:CARD=ALSA,DEV=1
    bcm2835 ALSA, bcm2835 IEC958/HDMI
    Direct hardware device without any conversions
plughw:CARD=ALSA,DEV=0
    bcm2835 ALSA, bcm2835 ALSA
    Hardware device with all software conversions
plughw:CARD=ALSA,DEV=1
    bcm2835 ALSA, bcm2835 IEC958/HDMI
    Hardware device with all software conversions
sysdefault:CARD=seeed8micvoicec
    seeed-8mic-voicecard,
    Default Audio Device
dmix:CARD=seeed8micvoicec,DEV=0
    seeed-8mic-voicecard,
    Direct sample mixing device
dsnoop:CARD=seeed8micvoicec,DEV=0
    seeed-8mic-voicecard,
    Direct sample snooping device
hw:CARD=seeed8micvoicec,DEV=0
    seeed-8mic-voicecard,
    Direct hardware device without any conversions
plughw:CARD=seeed8micvoicec,DEV=0
    seeed-8mic-voicecard,
    Hardware device with all software conversions

Step 3. Record and Play

You can record then play, or you can record and play at the same time.

#It will capture sound on AC108 and save as a.wav
arecord -Dac108 -f S32_LE -r 16000 -c 8 a.wav
#Take care of that the captured mic audio is on the first 6 channels

#It will play sound file a.wav on AC101
aplay -D ac101 a.wav
#Do not use -D plughw:1,0 directly except your wave file is single channel only.

#Doing capture && playback the same time
arecord -D hw:1,0 -f S32_LE -r 16000 -c 8 to_be_record.wav &
#mono_to_play.wav is a mono channel wave file to play
aplay -D plughw:1,0 -r 16000 mono_to_play.wav

Note

Limit for developer using 4-Mic linear Array Kit(or 4-Mic Linear Array Kit) doing capture & playback the same time:

-1. capture must be start first, or else the capture channels will possibly be disorder.

-2. playback output channels must fill with 8 same channels data or 4 same stereo channels data, or else the speaker or headphone will output nothing possibly.

-3. If you want to play and record at the same time, the aplay music file must be mono, or you can not use this command to play.

Also you can play and record with Audacity.

Tips

You should open Audacity via VNC or you can just use a monitor to open it

$ sudo apt update
$ sudo apt install audacity
$ audacity                      // run audacity

Play with Voice assistant

Step 1. Set the python-virtualenv environment

sudo apt install python-virtualenv          # install python virtualenv tool
virtualenv --system-site-packages ~/env     # create a virtual python environment
source ~/env/bin/activate                   # activate the virtual environment
(env) pi@raspberrypi:~ $

Step 2. Install AVS

(env) pi@raspberrypi:~ $ cd ~/
(env) pi@raspberrypi:~ $ git clone https://github.com/respeaker/avs
(env) pi@raspberrypi:~ $ cd avs    # install Requirements
(env) pi@raspberrypi:~ $ python setup.py install 
(env) pi@raspberrypi:~ $ sudo apt-get install gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gir1.2-gstreamer-1.0 python-gi python-gst-1.0
(env) pi@raspberrypi:~ $ sudo apt remove gstreamer1.0-omx gstreamer1.0-omx-rpi

Step 3. Get Alexa or Baidu authorization

To get the authorization, you need to open the Internet Brownser to log in your Amazon or Baidu ID, so you need to use VNC Viewer or just do it via monitor and keyboard. The same as ssh, you need the IP of your raspberry to log in VNC.

Play with Alexa

pi@raspberrypi:~ $ source ~/env/bin/activate
(env) pi@raspberrypi:~ $ alexa-auth

When you log in successfully, you can tap the following command to run Alexa

pi@raspberrypi:~ $ source ~/env/bin/activate
(env) pi@raspberrypi:~ $ alexa-tap

Then you can tap the Enter key to ask Alexa question and talk with Alexa.

Play with Dueros

If we want to switch between alexa-auth and dueros-auth, please delete /home/pi/.avs.json first. The file is hidden and use the ls -la to list the file.

rm -f /home/pi/.avs.json

Then you can use the command in VNC to log in Baidu ID

pi@raspberrypi:~ $ source ~/env/bin/activate
(env) pi@raspberrypi:~ $ dueros-auth

When you log in successfully, you can tap the following command to run Dueros.

pi@raspberrypi:~ $ source ~/env/bin/activate
(env) pi@raspberrypi:~ $ alexa-tap

Play with Snowboy

To do this part, you also need to get the authorization of Alexa or Baidu at first.

As you may see, the demos above trigger the Alexa or Dueros by tapping the Enter key. What if you want to wake up the Alexa by voice? Well, you can use snowboy. And you only need simple steps to make that happen.

Step 1. Install Snowboy

cd ~
git clone https://github.com/respeaker/4mics_hat.git
source ~/env/bin/activate              # activate the virtual, if we have already activated, skip this step
(env) pi@raspberrypi:~ $ cd ~/4mics_hat
(env) pi@raspberrypi:~/4mics_hat $ sudo apt install libatlas-base-dev     # install snowboy dependencies
(env) pi@raspberrypi:~/4mics_hat $ sudo apt install python-pyaudio        # install pyaudio
(env) pi@raspberrypi:~/4mics_hat $ pip install ./snowboy*.whl             # install snowboy for KWS
(env) pi@raspberrypi:~/4mics_hat $ pip install ./webrtc*.whl              # install webrtc for DoA
(env) pi@raspberrypi:~/4mics_hat $ cd ~/
(env) pi@raspberrypi:~ $ git clone https://github.com/voice-engine/voice-engine
(env) pi@raspberrypi:~ $ cd voice-engine/
(env) pi@raspberrypi:~/voice-engine $ python setup.py install

Step 2. Configure Pulse Audio

cd ~
sudo apt install pulseaudio
cd seeed-voicecard
cd pulseaudio
cd pulse_config_6mic
sudo cp seeed-voicecard.conf /usr/share/pulseaudio/alsa-mixer/profile-sets/
Then you need to edit the rules. As the system start, when the card "seeed8micvoicec" is detected, the PULSE_PROFILE_SET variable will be set in the udev database, and PulseAudio will be forced to use seeed-voicecard.conf. At first, please use the following command to check the rule.
sudo nano /lib/udev/rules.d/90-pulseaudio.rules
Then add the following lines at about line 87(behind the setting for some laptops and before the line GOTO="pulseaudio_end")
# Seeed Voicecard
ATTR{id}=="seeed8micvoicec",ATTR{number}=="1",ENV{PULSE_PROFILE_SET}="seeed-voicecard.conf"
It should be like: Then presss Ctrl+X to quite, and tap Y to save the modification you've just made. The value of ATTR{number} can be found with command:
udevadm info -a -p /sys/class/sound/card1/:

Step 3. config default.pa and daemon.conf

sudo cp default.pa /etc/pulse/
sudo cp daemon.conf /etc/pulse/

Step 4. reboot raspberry pi and check

sudo reboot
pulseaudio --start  # start pulse at first
pactl info  # check the setting

# The output should be like this
# You could see the default sink is seeed-2ch and default source is seeed-8ch
pi@raspberrypi:~ $ pactl info
Server String: /run/user/1000/pulse/native
Library Protocol Version: 32
Server Protocol Version: 32
Is Local: yes
Client Index: 6
Tile Size: 65496
User Name: pi
Host Name: raspberrypi
Server Name: pulseaudio
Server Version: 10.0
Default Sample Specification: s32le 8ch 96000Hz
Default Channel Map: front-left,front-left-of-center,front-center,front-right,front-right-of-center,rear-center,aux0,aux1
Default Sink: alsa_output.platform-soc_sound.seeed-2ch
Default Source: alsa_input.platform-soc_sound.seeed-8ch
Cookie: 3523:e5af

After you configure this snowboy, please do the following:

source ~/env/bin/activate 
cd ~/voice-engine/examples
python kws_alexa_for_4mic_liner_pihat.py
Then you will see the LEDs light up, and you can call Snowboy to wake it up.

FAQ

Q1: There are only 4 Mic in the Mic Array, how could it be 8 channels?

A1: There are 2 AC108 in this array, and each AC108 chip has 4 channel output. So it comes a total of 8 channels here, 4 of which are for the microphone, two channels for the playback and the the rest 2 channels are not used.

Resources

Tech Support

Please do not hesitate to contact techsupport@seeed.cc if you have any technical issue. Or submit the issue into our forum.