0
  • 聊天消息
  • 系统消息
  • 评论与回复
登录后你可以
  • 下载海量资料
  • 学习在线课程
  • 观看技术视频
  • 写文章/发帖/加入社区
创作中心

完善资料让更多小伙伴认识你,还能领取20积分哦,立即完善>

3天内不再提示

发烧友实测 | i.MX8MP 编译DPDK源码实现rte_ring无锁环队列进程间通信

飞凌嵌入式 2022-01-10 16:29 次阅读

ad6972f6-6fd6-11ec-86cf-dac502259ad0.jpg

作者|donatello1996

来源 | 电子发烧友

题图|飞凌嵌入式

rte_ring是一个用CAS实现的无锁FIFO环形队列,支持多消费者/生产者同时出入队列,常用于多线程/多进程之间的通信。具体原理可以查看DPDK官方文档或者阅读源码,本文采用的硬件板卡为飞凌嵌入式OKMX8MP-C开发板,系统版本Linux5.4.70+Qt5.15.0,主要介绍通过编译DPDK源码实现rte_ring无锁环队列进程间通信

下面就跟着小编一起了解下。

一、内核编译

下载并解压飞凌厂商提供的iMX8MP内核源码压缩包分卷:

ad94f48a-6fd6-11ec-86cf-dac502259ad0.jpg

在虚拟机中合并压缩分卷并解压得出内核源码包文件夹OK8MP-linux-kernel,将文件夹使用tar打包并复制到TF卡文件系统中解压:

adcb86f8-6fd6-11ec-86cf-dac502259ad0.jpg

找到内核源码中的配置文件OK8MP-C_defconfig:

ade9250a-6fd6-11ec-86cf-dac502259ad0.jpg

这个就是make选项,使用

make OK8MP-C_defconfig

指令即可配置编译选项:

make -j4

开始编译:

注意开始编译前需要安装常用软件:

apt install bison bc flex

ae1a54cc-6fd6-11ec-86cf-dac502259ad0.jpg

ae5d461a-6fd6-11ec-86cf-dac502259ad0.jpg

增量编译完毕:

ae99b79e-6fd6-11ec-86cf-dac502259ad0.jpg

二、DPDK编译

接下来就可以下载DPDK并运行rte_ring无锁环队列Demo应用,需要从

https://www.dpdk.org/

官网中下载DPDK 19.11.10 (LTS)长期支持版本:

aed037f6-6fd6-11ec-86cf-dac502259ad0.jpg

在根目录下的mk/文件夹下找到名为rte_vars.mk设置文件,找到环境变量RTE_KERNELDIR,修改为上述的内核源码路径:

aef06c42-6fd6-11ec-86cf-dac502259ad0.jpg

af1baf2e-6fd6-11ec-86cf-dac502259ad0.jpg

RTE_KERNELDIR ?= /home/OK8MP-linux-kernel/

进入usertools文件夹,找到dpdk-setup.sh脚本并运行

af50d12c-6fd6-11ec-86cf-dac502259ad0.jpg

afa77ef0-6fd6-11ec-86cf-dac502259ad0.jpg

选择8,ARM64-armv8a-linuxapp-gcc,

aff63860-6fd6-11ec-86cf-dac502259ad0.jpg

这个选项会使dpdk的gcc交叉编译链生成适用于armv8a处理器的外部库,外部库中有kmod和lib等ko文件和so文件,是用于第三方程序开发和运行的:

b02ae736-6fd6-11ec-86cf-dac502259ad0.jpg

b062a59a-6fd6-11ec-86cf-dac502259ad0.jpg

使用指令

insmod /home/dpdk-stable-19.11.10/arm64-armv8a-linuxapp-gcc/kmod/igb_uio.ko
左右滑动查看完整代码

加载igb_uio.ko驱动文件,这是进行dpdk开发必备的步骤:

b09bad04-6fd6-11ec-86cf-dac502259ad0.jpg

然后是使用dpdk-devbind.py脚本手动进行hugepage大页内存绑定,此处为numa方式:

b0c8c2ee-6fd6-11ec-86cf-dac502259ad0.jpg

此举会将/mnt/huge文件mount成hugepage映射文件,并实实在在地占用内存空间:

b104b54c-6fd6-11ec-86cf-dac502259ad0.jpg
b12e5848-6fd6-11ec-86cf-dac502259ad0.jpg

三、rte_ring无锁环队列

准备工作完成,我们接下来可以进行rte_ring无锁环队列Demo代码的编写,但是在编写之前,需要对无锁环队列有一个基本的认识:https://blog.csdn.net/chen98765432101/article/details/69367633


无论是dpdk第三方开发的rte_ring还是Linux内核中本就存在的无锁环队列,其基本原理类似,在一条分配好的队列型内存空间中,读写方式为FIFO(先进先出),读和写的动作分别有两个进程或两个线程进行,写进程不断往地址自增的内存位置写入数据,读进程不断读取地址自增的内存位置的数据,当写位置的内存地址已为队列中内存的最高值时,需要释放队列中内存地址最低值的空间供写进程继续写,方式仍与上一周期相同(不断往地址自增的内存位置写入数据),释放过程需要保证对末尾内存地址空间的锁定与解锁,避免读写过程出错。而不同的是,Linux内核中的无锁环队列,地址管理和读写控制均由内核进行,而dpdk的rte_ring则由dpdk内部的控制器进行,因为dpdk这一模块完整地接管了所分配内存空间的管理权,是直接绕过Linux内核进行管理的,内核也无权访问dpdk控制器的具体管理细节。

b166ff54-6fd6-11ec-86cf-dac502259ad0.jpg

b1bab982-6fd6-11ec-86cf-dac502259ad0.jpg

b1f20900-6fd6-11ec-86cf-dac502259ad0.jpg

b2256d7c-6fd6-11ec-86cf-dac502259ad0.jpg

编写无锁环队列两个进程的Demo,先写Primary进程:

#include #include #include #include #include #include #include #include #include #include 
#include #include 
#define RTE_LOGTYPE_APP RTE_LOGTYPE_USER1
static const char *_MSG_POOL = "MSG_POOL";static const char *_SEC_2_PRI = "SEC_2_PRI";static const char *_PRI_2_SEC = "PRI_2_SEC";static const char *_PRI_2_THI = "PRI_2_THI";
struct rte_ring *send_ring, *recv_ring , *send_ring_third;struct rte_mempool *message_pool;volatile int quit = 0;
static void * lcore_recv(void *arg){        unsigned lcore_id = rte_lcore_id();
        printf("Starting core %u\n", lcore_id);        while (!quit){                void *msg;                if (rte_ring_dequeue(recv_ring, &msg) < 0)                {                        usleep(5);                        continue;                }                printf("lcore_id = %d Received '%s'\n" , lcore_id , (char *)msg);                rte_mempool_put(message_pool , msg);        }
        return 0;}
int string_size = 100;int elt_size = 128;pthread_t id1;
int main(int argc, char **argv){        const unsigned flags = 0;        const unsigned ring_size = 64;        const unsigned pool_size = 1024;        const unsigned pool_cache = 32;        const unsigned priv_data_sz = 0;
        int ret;        unsigned lcore_id;
        ret = rte_eal_init(argc, argv);        if (ret < 0)                rte_exit(EXIT_FAILURE, "Cannot init EAL\n");
        send_ring = rte_ring_create(_PRI_2_SEC, ring_size, rte_socket_id(), flags);        recv_ring = rte_ring_create(_SEC_2_PRI, ring_size, rte_socket_id(), flags);        send_ring_third = rte_ring_create(_PRI_2_THI, ring_size, rte_socket_id(), flags);        message_pool = rte_mempool_create(_MSG_POOL, pool_size,                        elt_size, pool_cache, priv_data_sz,                        NULL, NULL, NULL, NULL,                        rte_socket_id(), flags);
        if (send_ring == NULL)                rte_exit(EXIT_FAILURE, "Problem getting sending ring\n");
        if (recv_ring == NULL)                rte_exit(EXIT_FAILURE, "Problem getting receiving ring\n");
        if (send_ring_third == NULL)                rte_exit(EXIT_FAILURE, "Problem getting send_ring_third\n");
        if (message_pool == NULL)                rte_exit(EXIT_FAILURE, "Problem getting message pool\n");
        pthread_create(&id1 , NULL , lcore_recv , NULL);        while(1)        {                void *msg = NULL;                if (rte_mempool_get(message_pool, &msg) < 0)                        continue;
                snprintf((char *)msg, string_size, "%s", "primary to secondary");                if (rte_ring_enqueue(send_ring , msg) < 0)                 {                        rte_mempool_put(message_pool, msg);                }
                if (rte_mempool_get(message_pool, &msg) < 0)                        continue;
                snprintf((char *)msg, string_size, "%s", "primary to third");                if (rte_ring_enqueue(send_ring_third , msg) < 0)                 {                        rte_mempool_put(message_pool, msg);                }
                sleep(1);        }
        return 0;}
左右滑动查看完整代码

注意在Makefile文件里面要关闭WERROR相关编译选项:

#   BSD LICENSE##   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.#   All rights reserved.##   Redistribution and use in source and binary forms, with or without#   modification, are permitted provided that the following conditions#   are met:##     * Redistributions of source code must retain the above copyright#       notice, this list of conditions and the following disclaimer.#     * Redistributions in binary form must reproduce the above copyright#       notice, this list of conditions and the following disclaimer in#       the documentation and/or other materials provided with the#       distribution.#     * Neither the name of Intel Corporation nor the names of its#       contributors may be used to endorse or promote products derived#       from this software without specific prior written permission.##   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
ifeq ($(RTE_SDK),)$(error "Please define RTE_SDK environment variable")endif
# Default target, can be overridden by command line or environmentRTE_TARGET ?= arm64-armv8a-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
# binary nameAPP = rte_ring_primary
# all source are stored in SRCS-ySRCS-y := main.c
CFLAGS += -O0CFLAGS += 
include $(RTE_SDK)/mk/rte.extapp.mk
左右滑动查看完整代码

Secondary进程:

#include #include #include #include #include #include #include #include #include #include 
#include #include 
#define RTE_LOGTYPE_APP RTE_LOGTYPE_USER1
static const char *_MSG_POOL = "MSG_POOL";static const char *_SEC_2_PRI = "SEC_2_PRI";static const char *_PRI_2_SEC = "PRI_2_SEC";
struct rte_ring *send_ring, *recv_ring;struct rte_mempool *message_pool;volatile int quit = 0;int string_size = 100;
static int lcore_send(__attribute__((unused)) void *arg){        unsigned lcore_id = rte_lcore_id();
        while(1)        {                void *msg = NULL;                if (rte_mempool_get(message_pool, &msg) < 0)                        continue;
                snprintf((char *)msg , string_size , "%s", "secondary to primary");                if (rte_ring_enqueue(send_ring , msg) < 0)                 {                        rte_mempool_put(message_pool, msg);                }                sleep(1);        }        return 0;}
pthread_t id1;
int main(int argc, char **argv){        const unsigned flags = 0;        const unsigned ring_size = 64;        const unsigned pool_size = 1024;        const unsigned pool_cache = 32;        const unsigned priv_data_sz = 0;
        int ret;        unsigned lcore_id;        ret = rte_eal_init(argc, argv);        if (ret < 0)                rte_exit(EXIT_FAILURE, "Cannot init EAL\n");
        recv_ring = rte_ring_lookup(_PRI_2_SEC);        send_ring = rte_ring_lookup(_SEC_2_PRI);        message_pool = rte_mempool_lookup(_MSG_POOL);
        if (send_ring == NULL)                rte_exit(EXIT_FAILURE, "Problem getting sending ring\n");        if (recv_ring == NULL)                rte_exit(EXIT_FAILURE, "Problem getting receiving ring\n");        if (message_pool == NULL)                rte_exit(EXIT_FAILURE, "Problem getting message pool\n");
        pthread_create(&id1 , NULL , lcore_send , NULL);    while (1)        {                lcore_id = rte_lcore_id();        void * msg = NULL;        if (rte_ring_dequeue(recv_ring, &msg) < 0)                {            usleep(5);            continue;        }
        printf("lcore_id = %d Received: %s\n" , lcore_id , (char *)msg);
        rte_mempool_put(message_pool, msg);    }
        return 0;}

左右滑动查看完整代码

同样在Makefile文件里面要关闭WERROR相关编译选项:

#   BSD LICENSE##   Copyright(c) 2010-2014 Intel Corporation. All rights reserved.#   All rights reserved.##   Redistribution and use in source and binary forms, with or without#   modification, are permitted provided that the following conditions#   are met:##     * Redistributions of source code must retain the above copyright#       notice, this list of conditions and the following disclaimer.#     * Redistributions in binary form must reproduce the above copyright#       notice, this list of conditions and the following disclaimer in#       the documentation and/or other materials provided with the#       distribution.#     * Neither the name of Intel Corporation nor the names of its#       contributors may be used to endorse or promote products derived#       from this software without specific prior written permission.##   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
ifeq ($(RTE_SDK),)$(error "Please define RTE_SDK environment variable")endif
# Default target, can be overridden by command line or environmentRTE_TARGET ?= arm64-armv8a-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
# binary nameAPP = rte_ring_secondary
# all source are stored in SRCS-ySRCS-y := main.c
CFLAGS += -O3CFLAGS += $()
include $(RTE_SDK)/mk/rte.extapp.mk
左右滑动查看完整代码

运行,这里说一下,基于rte_ring的进程间通信,Secondary进程最好是使用auto类型:

./rte_ring_primary --proc-type primary./rte_ring_secondary --proc-type auto

运行效果:

b254fb0a-6fd6-11ec-86cf-dac502259ad0.jpg

b2a99c3c-6fd6-11ec-86cf-dac502259ad0.jpg

作者简介

donatello1996,某大型企业资深嵌入式工程师,电子发烧友论坛技术大牛,同时也是飞凌嵌入式多年铁粉,曾基于飞凌多款板卡产出过优质测评文章或使用心得。本期三篇文章为donatello1996在使用OKMX8MP-C开发板过程中精心产出的干货,在此对donatello1996表示感谢。

声明:本文内容及配图由入驻作者撰写或者入驻合作网站授权转载。文章观点仅代表作者本人,不代表电子发烧友网立场。文章及其配图仅供工程师学习之用,如有内容侵权或者其他违规问题,请联系本站处理。 举报投诉
  • 嵌入式
    +关注

    关注

    4963

    文章

    18161

    浏览量

    286545
收藏 人收藏

    评论

    相关推荐

    i.MX8MP开发板上实现SPI转CAN

    转CAN芯片型号为:MCP2518,这款芯片可转出CAN-FD,若只需要CAN功能,可参照该方式移植MCP2515或其他芯片; 本次移植的MCP2518芯片驱动来源是i.MX8MQ的源码(处理器
    发表于 09-11 15:26

    u-boot以i.MX8MP启动时,boot device是根据ROM中Boot_SW_Info的Boot Device Type来确定的吗?

    你好。 我有两个问题。 Q1:u-boot以i.MX8MP启动时,boot device是根据ROM中Boot_SW_Info的Boot Device Type来确定的吗? Q2
    发表于 06-08 08:41

    如何扩展imx8mp.dtsi中的i.mx8mp cpufreq设置以降低cpu温度?

    一、基本信息: i.mx8mp ,Linux5.10.35, Yocto hardknott 。 i.mx8mp 被设计为数据采集设备。 2、问题: 如何扩展 imx8mp.dtsi 中
    发表于 05-18 13:25

    i.MX8MP PCI问题如何解决?

    我已经购买了 Variscite 评估套件,所以我同时使用了 VAR-SOM-MX8M-PLUS 和 Symphony 板。 我正在使用 Debian Bullseye 操作系统。图片
    发表于 05-18 06:33

    i.mx8mp是否意味着每个通道都可以支持768KHz/32位?

    i.mx8mp RM 文档中,P5944,我对 SAI 容量有些怀疑。 这是否意味着每个通道或通道都可以支持 768KHz/32 位,或者 SAI2/SAI5 模块总共可以支持 768KHz/32 位?
    发表于 05-04 07:04

    i.MX8MP能够解码来自8MP USB摄像头的MJPEG视频吗?

    我想确认 i.MX8MP 是否能够解码来自 8MP USB 摄像头的 MJPEG 视频并将其编码为 H.265。
    发表于 04-27 06:57

    DPDK 22.11.1 LTS应用程序对rte_pktmbuf_alloc的调用失败怎么解决?

    从输入队列接收 mbuf 队列,并处理每个 mbuf: 2. 使用 rte_pktmbuf_alloc 为 mbuf 分配一个尾部。此分配发生在用于分配接收到的数据包的同一池中。 3.
    发表于 04-23 09:00

    i.MX8MP,USB TYPE C,gatesgarth工作正常吗?

    i.MX8MP,USB TYPE C,gatesgarth,工作正常吗?
    发表于 04-21 08:47

    LS1046A rte_vdev_init上的DPDK因crypto_dpaa_sec而失败怎么解决?

    嗨,我正在尝试使用 LS1046A 进行 DPDK 密码加密,但发现对 rte_vdev_init 的调用失败返回 “没有这样的文件或目录”。我在调用 rte_vdev_init 时请求
    发表于 04-20 06:53

    Codeaurora迁移到其他平台后,yocto编译i.MX Linux Yocto Project BSP 5.10.9_1.0.0 )失败怎么解决?

    在Codeaurora迁移到其他平台后,我们目前开发i.mx8mp 使用的BSP(i.MX Linux Yocto Project BSP 5.10.9_1.0.0)无法编译了。 有人能帮忙给出一个解决方案,简单快速地修复一下,
    发表于 04-10 07:28

    memtool是否支持i.MX8MP

    你好我正在使用 SC2310H 将数据输出到 i.MX8MP 的 MIPI,但数据没有到达 VB2 队列。我想参考以下寄存器值来检查 i.MX8MP 是否可以接收 MIPI 数据。32E4_0020
    发表于 04-06 07:39

    如何使用TRACE32调试器调试i.MX8MP uboot?

    本文档介绍如何使用 TRACE32 调试器调试 i.MX8MP uboot。1.搭建uboot按照IMX_LINUX_USERS_GUIDE.pdf搭建uboot,生成flash.bin。我们将从
    发表于 04-06 06:50

    i.MX8MP/Linux 5.4.70.2.3.0 HDMI热插拔显示异常是怎么回事?

    平台:i.MX8MP EVKBSP:Linux 5.4.70.2.3.0客户报告在运行 L5.4.70.2.3.0 的 i.MX8MP 上进行 HDMI 热插拔时,HDMI 显示器上的 UI 异常并且左侧有更多线条。
    发表于 04-04 07:48

    i.mx8mp移植wm8962芯片报错的原因?如何解决?

    nxp工程师工程师:我们我们的i.mx8mp wm8962b芯片。我们将设备树中wm8960相关点修改为wm8962,将内芯中自带的wm8962.c驱动打开编译。设备树截取如下:sai3的配置和内核
    发表于 04-03 07:18

    i.MX8MP EQOS MAC_Ingress_Timestamp_Latency和MAC_Egress_Timestamp_Latency始终为0的原因?

    i.MX8MP 处理器参考手册表明 1588 时间戳入口和出口校正必须加载到 MAC_Timestamp_[Ingress,Egress]_Corr_* 寄存器(参考手册第 11.7.2.5.3.1
    发表于 04-03 07:15