基于麦克风阵列的声源定位_python播放声音模块

基于麦克风阵列的声源定位_python播放声音模块上一篇文章说到odas_web界面非常难安装,并且运行也很卡。所以我自己用python写了一个界面程序,用来接收odas处理完的结果。这个界面程序与odas之间是通过socket连接的,界面作为服务器,odas作为客户端,由于有两路数据,所以各有两个服务器和客户端。但是实际绘制在界面上的是SSL的结果,不是SST的结果。其实我也试过SST的结果,从直观的感受而言,效果会比SSL差一些,实时性不是很高,我的理解SST的好处是可以跟踪音源是否有活动。

大家好,又见面了,我是你们的朋友全栈君。如果您正在找激活码,请点击查看最新教程,关注关注公众号 “全栈程序员社区” 获取激活教程,可能之前旧版本教程已经失效.最新Idea2022.1教程亲测有效,一键激活。

Jetbrains全系列IDE使用 1年只要46元 售后保障 童叟无欺

上一篇文章说到odas_web界面非常难安装,并且运行也很卡。所以我自己用python写了一个界面程序,用来接收odas处理完的结果。

这个界面程序与odas之间是通过socket连接的, 界面作为服务器,odas作为客户端,由于有两路数据,所以各有两个服务器和客户端。但是实际绘制在界面上的是SSL的结果,不是SST的结果。其实我也试过SST的结果,从直观的感受而言,效果会比SSL差一些,实时性不是很高,我的理解SST的好处是可以跟踪音源是否有活动。 

另外,我也试过把这个代码在树莓派3上跑,性能会比odas_web好不少,但是还是不如笔记本电脑跑得流畅。在树莓派上先要装python3-opencv,然后用python3来运行这个界面程序。

安装命令和两个程序的运行命令可以参考如下:

sudo apt install python3-opencv
python3 DOA_sound.py
./bin/odaslive -c config/odaslive/shao.cfg

视频:

麦克风阵列 声源定位 定向拾音_哔哩哔哩_bilibili基于麦克风阵列的声源定位_python播放声音模块https://www.bilibili.com/video/BV1xu411B7s3

下面附上我写的界面程序:

#!/usr/bin/env python
import socket
import sys
import threading
import random
import os
import time
import struct
import cv2
import signal
import json
import ast
import numpy as np

stop = False
HOST = "0.0.0.0"
PORT = 9000
SOCK_ADDR = (HOST, PORT)

PORT2 = 9001
SOCK_ADDR2 = (HOST, PORT2)


def stop_handler(signum, frame):
    global running
    running = False

signal.signal(signal.SIGINT, stop_handler) 

spectrum_rgb3_lut = [
	[   0,   0,   0 ],
	[   0,   0,   3 ],
	[   0,   0,   6 ],
	[   0,   0,   9 ],
	[   0,   0,  12 ],
	[   0,   0,  15 ],
	[   0,   0,  18 ],
	[   0,   0,  21 ],
	[   0,   0,  24 ],
	[   0,   0,  27 ],
	[   0,   0,  30 ],
	[   0,   0,  33 ],
	[   0,   0,  36 ],
	[   0,   0,  39 ],
	[   0,   0,  42 ],
	[   0,   0,  45 ],
	[   0,   0,  48 ],
	[   0,   0,  51 ],
	[   0,   0,  54 ],
	[   0,   0,  57 ],
	[   0,   0,  60 ],
	[   0,   0,  63 ],
	[   0,   0,  66 ],
	[   0,   0,  69 ],
	[   0,   0,  72 ],
	[   0,   0,  75 ],
	[   0,   0,  78 ],
	[   0,   0,  81 ],
	[   0,   0,  84 ],
	[   0,   0,  87 ],
	[   0,   0,  90 ],
	[   0,   0,  93 ],
	[   0,   0,  96 ],
	[   0,   0,  99 ],
	[   0,   0, 102 ],
	[   0,   0, 105 ],
	[   0,   0, 108 ],
	[   0,   0, 111 ],
	[   0,   0, 114 ],
	[   0,   0, 117 ],
	[   0,   0, 120 ],
	[   0,   0, 123 ],
	[   0,   0, 126 ],
	[   0,   0, 129 ],
	[   0,   0, 132 ],
	[   0,   0, 135 ],
	[   0,   0, 138 ],
	[   0,   0, 141 ],
	[   0,   0, 144 ],
	[   0,   0, 147 ],
	[   0,   0, 150 ],
	[   0,   0, 153 ],
	[   0,   0, 156 ],
	[   0,   0, 159 ],
	[   0,   0, 162 ],
	[   0,   0, 165 ],
	[   0,   0, 168 ],
	[   0,   0, 171 ],
	[   0,   0, 174 ],
	[   0,   0, 177 ],
	[   0,   0, 180 ],
	[   0,   0, 183 ],
	[   0,   0, 186 ],
	[   0,   0, 189 ],
	[   0,   0, 192 ],
	[   0,   0, 195 ],
	[   0,   0, 198 ],
	[   0,   0, 201 ],
	[   0,   0, 204 ],
	[   0,   0, 207 ],
	[   0,   0, 210 ],
	[   0,   0, 213 ],
	[   0,   0, 216 ],
	[   0,   0, 219 ],
	[   0,   0, 222 ],
	[   0,   0, 225 ],
	[   0,   0, 228 ],
	[   0,   0, 231 ],
	[   0,   0, 234 ],
	[   0,   0, 237 ],
	[   0,   0, 240 ],
	[   0,   0, 243 ],
	[   0,   0, 246 ],
	[   0,   0, 249 ],
	[   0,   0, 252 ],
	[   0,   0, 255 ],
	[   0,   3, 252 ],
	[   0,   6, 249 ],
	[   0,   9, 246 ],
	[   0,  12, 243 ],
	[   0,  15, 240 ],
	[   0,  18, 237 ],
	[   0,  21, 234 ],
	[   0,  24, 231 ],
	[   0,  27, 228 ],
	[   0,  30, 225 ],
	[   0,  33, 222 ],
	[   0,  36, 219 ],
	[   0,  39, 216 ],
	[   0,  42, 213 ],
	[   0,  45, 210 ],
	[   0,  48, 207 ],
	[   0,  51, 204 ],
	[   0,  54, 201 ],
	[   0,  57, 198 ],
	[   0,  60, 195 ],
	[   0,  63, 192 ],
	[   0,  66, 189 ],
	[   0,  69, 186 ],
	[   0,  72, 183 ],
	[   0,  75, 180 ],
	[   0,  78, 177 ],
	[   0,  81, 174 ],
	[   0,  84, 171 ],
	[   0,  87, 168 ],
	[   0,  90, 165 ],
	[   0,  93, 162 ],
	[   0,  96, 159 ],
	[   0,  99, 156 ],
	[   0, 102, 153 ],
	[   0, 105, 150 ],
	[   0, 108, 147 ],
	[   0, 111, 144 ],
	[   0, 114, 141 ],
	[   0, 117, 138 ],
	[   0, 120, 135 ],
	[   0, 123, 132 ],
	[   0, 126, 129 ],
	[   0, 129, 126 ],
	[   0, 132, 123 ],
	[   0, 135, 120 ],
	[   0, 138, 117 ],
	[   0, 141, 114 ],
	[   0, 144, 111 ],
	[   0, 147, 108 ],
	[   0, 150, 105 ],
	[   0, 153, 102 ],
	[   0, 156,  99 ],
	[   0, 159,  96 ],
	[   0, 162,  93 ],
	[   0, 165,  90 ],
	[   0, 168,  87 ],
	[   0, 171,  84 ],
	[   0, 174,  81 ],
	[   0, 177,  78 ],
	[   0, 180,  75 ],
	[   0, 183,  72 ],
	[   0, 186,  69 ],
	[   0, 189,  66 ],
	[   0, 192,  63 ],
	[   0, 195,  60 ],
	[   0, 198,  57 ],
	[   0, 201,  54 ],
	[   0, 204,  51 ],
	[   0, 207,  48 ],
	[   0, 210,  45 ],
	[   0, 213,  42 ],
	[   0, 216,  39 ],
	[   0, 219,  36 ],
	[   0, 222,  33 ],
	[   0, 225,  30 ],
	[   0, 228,  27 ],
	[   0, 231,  24 ],
	[   0, 234,  21 ],
	[   0, 237,  18 ],
	[   0, 240,  15 ],
	[   0, 243,  12 ],
	[   0, 246,   9 ],
	[   0, 249,   6 ],
	[   0, 252,   3 ],
	[   0, 255,   0 ],
	[   3, 252,   0 ],
	[   6, 249,   0 ],
	[   9, 246,   0 ],
	[  12, 243,   0 ],
	[  15, 240,   0 ],
	[  18, 237,   0 ],
	[  21, 234,   0 ],
	[  24, 231,   0 ],
	[  27, 228,   0 ],
	[  30, 225,   0 ],
	[  33, 222,   0 ],
	[  36, 219,   0 ],
	[  39, 216,   0 ],
	[  42, 213,   0 ],
	[  45, 210,   0 ],
	[  48, 207,   0 ],
	[  51, 204,   0 ],
	[  54, 201,   0 ],
	[  57, 198,   0 ],
	[  60, 195,   0 ],
	[  63, 192,   0 ],
	[  66, 189,   0 ],
	[  69, 186,   0 ],
	[  72, 183,   0 ],
	[  75, 180,   0 ],
	[  78, 177,   0 ],
	[  81, 174,   0 ],
	[  84, 171,   0 ],
	[  87, 168,   0 ],
	[  90, 165,   0 ],
	[  93, 162,   0 ],
	[  96, 159,   0 ],
	[  99, 156,   0 ],
	[ 102, 153,   0 ],
	[ 105, 150,   0 ],
	[ 108, 147,   0 ],
	[ 111, 144,   0 ],
	[ 114, 141,   0 ],
	[ 117, 138,   0 ],
	[ 120, 135,   0 ],
	[ 123, 132,   0 ],
	[ 126, 129,   0 ],
	[ 129, 126,   0 ],
	[ 132, 123,   0 ],
	[ 135, 120,   0 ],
	[ 138, 117,   0 ],
	[ 141, 114,   0 ],
	[ 144, 111,   0 ],
	[ 147, 108,   0 ],
	[ 150, 105,   0 ],
	[ 153, 102,   0 ],
	[ 156,  99,   0 ],
	[ 159,  96,   0 ],
	[ 162,  93,   0 ],
	[ 165,  90,   0 ],
	[ 168,  87,   0 ],
	[ 171,  84,   0 ],
	[ 174,  81,   0 ],
	[ 177,  78,   0 ],
	[ 180,  75,   0 ],
	[ 183,  72,   0 ],
	[ 186,  69,   0 ],
	[ 189,  66,   0 ],
	[ 192,  63,   0 ],
	[ 195,  60,   0 ],
	[ 198,  57,   0 ],
	[ 201,  54,   0 ],
	[ 204,  51,   0 ],
	[ 207,  48,   0 ],
	[ 210,  45,   0 ],
	[ 213,  42,   0 ],
	[ 216,  39,   0 ],
	[ 219,  36,   0 ],
	[ 222,  33,   0 ],
	[ 225,  30,   0 ],
	[ 228,  27,   0 ],
	[ 231,  24,   0 ],
	[ 234,  21,   0 ],
	[ 237,  18,   0 ],
	[ 240,  15,   0 ],
	[ 243,  12,   0 ],
	[ 246,   9,   0 ],
	[ 249,   6,   0 ],
	[ 252,   3,   0 ],
	[ 255,   0,   0 ]]


class SocketClientObject(object):
    def __init__(self, socket, address ):
        self.socket = socket
        self.address = address

class ClientThread(threading.Thread):
    def __init__(self, client_object):
        threading.Thread.__init__(self)
        self.client_object = client_object

    def run(self):
        global running
        while running == True:
            img = np.zeros((800,800,3),np.uint8)
            data = self.client_object.socket.recv(1024)
            data = data.decode("utf-8")
            data = data.replace("\n", "")
            try:
                src = (data.split('[')[1]).split(']')[0]
                items = src.split(",        ")
                target = json.loads(items[0])
                x = int(float(target["x"]) * 400) + 400
                y = int(-float(target["y"]) * 400) + 400
                energy = int(float(target["E"]) * 255)
                if (energy > 80):
                    cv2.circle(img,  (x, y),  30,  (spectrum_rgb3_lut[255- energy][0], spectrum_rgb3_lut[255- energy][1], spectrum_rgb3_lut[255- energy][2]), -1)
                cv2.imshow('pu', img)
                if cv2.waitKey(1) & 0xFF == ord('q'):
                    break

            except:
                print "problem1"

        cv2.destroyAllWindows()
        self.client_object.socket.close()


class VideoThread(threading.Thread):
    def __init__(self,dest_object):
        threading.Thread.__init__(self)
        self.dest_object=dest_object

    def run(self):
        global running
        while running == True:
            #img = np.zeros((800,800,3),np.uint8)
            data = self.dest_object.socket.recv(1024)
            data = data.decode("utf-8")
            data = data.replace("\n", "")
            try:
                src = (data.split('[')[1]).split(']')[0]
                items = src.split(",        ")
                for item in items:
                    target = json.loads(item)
                    x = int(float(target["x"]) * 400) + 400
                    y = int(-float(target["y"]) * 400) + 400
                    activity = int(float(target["activity"]) * 255)
                    #if (activity > 100):
                    #    cv2.circle(img,  (x, y),  30, (0,255,0), -1)

                #cv2.imshow('pu2', img)
                #if cv2.waitKey(1) & 0xFF == ord('q'):
                    #break

            except:
                print "problem2"

        #cv2.destroyAllWindows()
        self.dest_object.socket.close()


def main():
    global running
    running = True

    try:
        sock1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock1.bind(SOCK_ADDR)
        sock1.listen(5)

        sock2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock2.bind(SOCK_ADDR2)
        sock2.listen(2)

        while running:
            (clientsocket, address) = sock1.accept()
            print " Accept client: ", address
            ct = ClientThread(SocketClientObject(clientsocket, address))
            ct.start()

            (dst,dst_addr) = sock2.accept()
	    print "Destination Connected by", dst_addr
            vt = VideoThread(SocketClientObject(dst,dst_addr))
	    vt.start()

    except:
        print "#! EXC: ", sys.exc_info()
        sock1.close()
        sock2.close()
        print "THE END! Goodbye!"

if __name__ == "__main__":
    main()

另外,我还附上我使用的shao.cfg文件:

# Configuration file for ReSpeaker USB 4 Mic Array (ReSpeaker USB Mic Array v2.0)

version = "2.1";

# Raw

raw: 
{

    fS = 16000;
    hopSize = 128;
    nBits = 16;
    nChannels = 6; 

    # Input with raw signal from microphones
    interface: {
        type = "soundcard";
        card = 2;
        device = 0;
    }

}

# Mapping

mapping:
{

    map: (2, 3, 4, 5);

}

# General

general:
{
    
    epsilon = 1E-20;

    size: 
    {
        hopSize = 128;
        frameSize = 256;
    };
    
    samplerate:
    {
        mu = 16000;
        sigma2 = 0.01;
    };

    speedofsound:
    {
        mu = 343.0;
        sigma2 = 25.0;
    };

    mics = (

        # Microphone 2
        { 
            mu = ( -0.032, +0.000, +0.000 ); 
            sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
            direction = ( +0.000, +0.000, +1.000 );
            angle = ( 80.0, 100.0 );
        },

        # Microphone 3
        { 
            mu = ( +0.000, -0.032, +0.000 ); 
            sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
            direction = ( +0.000, +0.000, +1.000 );
            angle = ( 80.0, 100.0 );
        },

        # Microphone 4
        { 
            mu = ( +0.032, +0.000, +0.000 ); 
            sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
            direction = ( +0.000, +0.000, +1.000 );
            angle = ( 80.0, 100.0 );
        },

        # Microphone 5
        { 
            mu = ( +0.000, +0.032, +0.000 ); 
            sigma2 = ( +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000, +0.000 );
            direction = ( +0.000, +0.000, +1.000 );
            angle = ( 80.0, 100.0 );        
        }
        
    );

    # Spatial filter to include only a range of direction if required
    # (may be useful to remove false detections from the floor)
    spatialfilters = (

        {
            direction = ( +0.000, +0.000, +1.000 );
            angle = (80.0, 100.0);

        }    

    );

    nThetas = 181;
    gainMin = 0.25;

};

# Stationnary noise estimation

sne:
{
    
    b = 3;
    alphaS = 0.1;
    L = 150;
    delta = 3.0;
    alphaD = 0.1;

}

# Sound Source Localization

ssl:
{

    nPots = 4;
    nMatches = 10;
    probMin = 0.5;
    nRefinedLevels = 1;
    interpRate = 4;

    # Number of scans: level is the resolution of the sphere
    # and delta is the size of the maximum sliding window
    # (delta = -1 means the size is automatically computed)
    scans = (
        { level = 2; delta = -1; },
        { level = 4; delta = -1; }
    );

    # Output to export potential sources
    potential: {

        # format = "undefined";
        format = "json";

        interface: {
            #type = "blackhole";
            type = "socket"; ip = "127.0.0.1"; port = 9000;
            #type = "terminal";
        };

    };

};

# Sound Source Tracking

sst:
{  

    # Mode is either "kalman" or "particle"

    mode = "kalman";

    # Add is either "static" or "dynamic"

    add = "dynamic";    

    # Parameters used by both the Kalman and particle filter

    active = (
        { weight = 1.0; mu = 0.4; sigma2 = 0.0025 }
    );

    inactive = (
        { weight = 1.0; mu = 0.25; sigma2 = 0.0025 }
    );

    sigmaR2_prob = 0.0025;
    sigmaR2_active = 0.0225;
    sigmaR2_target = 0.0025;
    Pfalse = 0.1;
    Pnew = 0.1;
    Ptrack = 0.8;

    theta_new = 0.9;
    N_prob = 5;
    theta_prob = 0.8;
    N_inactive = ( 250, 250, 250, 250 );
    theta_inactive = 0.9;

    # Parameters used by the Kalman filter only

    kalman: {

        sigmaQ = 0.001;
        
    };
   
    # Parameters used by the particle filter only

    particle: {

        nParticles = 1000;
        st_alpha = 2.0;
        st_beta = 0.04;
        st_ratio = 0.5;
        ve_alpha = 0.05;
        ve_beta = 0.2;
        ve_ratio = 0.3;
        ac_alpha = 0.5;
        ac_beta = 0.2;
        ac_ratio = 0.2;
        Nmin = 0.7;

    };

    target: ();

    # Output to export tracked sources
    tracked: {

        format = "json";

        interface: {
            #type = "file"; path = "tracks.txt";
            type = "socket"; ip = "127.0.0.1"; port = 9001;
            #type = "terminal";
        };

    };

}

sss:
{
    
    # Mode is either "dds", "dgss" or "dmvdr"

    mode_sep = "dds";
    mode_pf = "ms";

    gain_sep = 1.0;
    gain_pf = 10.0;

    dds: {

    };

    dgss: {

        mu = 0.01;
        lambda = 0.5;

    };

    dmvdr: {

    };

    ms: {

        alphaPmin = 0.07;
        eta = 0.5;
        alphaZ = 0.8;        
        thetaWin = 0.3;
        alphaWin = 0.3;
        maxAbsenceProb = 0.9;
        Gmin = 0.01;
        winSizeLocal = 3;
        winSizeGlobal = 23;
        winSizeFrame = 256;

    };

    ss: {

        Gmin = 0.01;
        Gmid = 0.9;
        Gslope = 10.0;

    };

    separated: {

        fS = 16000;
        hopSize = 128;
        nBits = 16;        

        interface: {
            type = "file";
            path = "separated.raw";
        };        

    };

    postfiltered: {

        fS = 16000;
        hopSize = 128;
        nBits = 16;        
        gain = 10.0;

        interface: {
            type = "file";
            path = "postfiltered.raw";
        };        

    };

};

classify:
{
    
    frameSize = 4096;
    winSize = 3;
    tauMin = 88;
    tauMax = 551;
    deltaTauMax = 20;
    alpha = 0.3;
    gamma = 0.05;
    phiMin = 0.5;
    r0 = 0.2;    

    category: {

        format = "undefined";

        interface: {
            type = "blackhole";
        }

    }

}

版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 举报,一经查实,本站将立刻删除。

发布者:全栈程序员-用户IM,转载请注明出处:https://javaforall.cn/193733.html原文链接:https://javaforall.cn

【正版授权,激活自己账号】: Jetbrains全家桶Ide使用,1年售后保障,每天仅需1毛

【官方授权 正版激活】: 官方授权 正版激活 支持Jetbrains家族下所有IDE 使用个人JB账号...

(0)


相关推荐

  • eigen库教程_mkl库

    eigen库教程_mkl库1.Matrix类:定义:Matrix<类型,行,列>eigen库中封装好了一些常用的矩阵,例如:typedefMatrix<float,4,4>Matrix4f;当然我们也可以自己设置,矩阵的行和列可以设置为固定的值也可以设置动态的(Dynamic),小的尺寸用固定的,大的尺寸用动态的,使用固定尺寸可以避免动态内存的开辟。1)初始化…

    2022年10月18日
  • JVM内存结构和Java内存模型别再傻傻分不清了

    JVM内存结构和Java内存模型别再傻傻分不清了JVM内存结构和Java内存模型都是面试的热点问题,名字看感觉都差不多,网上有些博客也都把这两个概念混着用,实际上他们之间差别还是挺大的。通俗点说,JVM内存结构是与JVM的内部存储结构相关,而Java内存模型是与多线程编程相关,本文针对这两个总是被混用的概念展开讲解。文章目录JVM内存结构JVM构成JVM内存结构程序计数器虚拟机栈本地方法栈堆方法区GC垃圾回收机制1.垃圾判别方法引用计数算法可达性分析算法2.垃圾回收算法标记清除法标记整理法复制算法3.分代垃圾回收机制4.垃圾回收器5.四种引

  • js 判断字符串中是否包含某个字符串的方法实例

    js 判断字符串中是否包含某个字符串的方法实例String对象的方法方法一:indexOf()(推荐)varstr=”123″;console.log(str.indexOf(“3”)!=-1);//trueindexOf()方法可返回某个指定的字符串值在字符串中首次出现的位置。如果要检索的字符串值没有出现,则该方法返回-1。方法二:search()varstr=…

  • 【机器学习】彻底理解神经网络中的反向传播算法(BP)

    【机器学习】彻底理解神经网络中的反向传播算法(BP)目录1,前言2,例程Step1前向传播Step2反向传播3,代码实现1,前言最近在看机器学习神经网络方面的知识,也看了很多关于反向传播算法原理的介绍,有一篇文章写的很好,在这里记录下,并且加入自己的理解。反向传播法其实是神经网络的基础了,但是很多人在学的时候总是会遇到一些问题,或者看到大篇的公式觉得好像很难就退缩了,其实不难,就是一个链式求导法则反复用。如果不想…

  • pytorch转tensorrt_pytorch是啥

    pytorch转tensorrt_pytorch是啥报错原因分析涉及到梯度传播的tensor变量不能够直接numpy()转numpy,需要使用:var.detach().numpy()

  • Burp Suite安装配置详解(附Java 环境安装)

    Burp Suite安装配置详解(附Java 环境安装)BurpSuite安装配置详解1.Java安装与环境配置详解1.0下载JavaSDK1.8最新版2.0配置Java环境变量功能快捷键合理的创建标题,有助于目录的生成如何改变文本的样式插入链接与图片如何插入一段漂亮的代码片生成一个适合你的列表创建一个表格设定内容居中、居左、居右SmartyPants创建一个自定义列表如何创建一个注脚注释也是必不可少的KaTeX数学公式新的甘特图功能,…

发表回复

您的电子邮箱地址不会被公开。

关注全栈程序员社区公众号