博客

  • 传输网技术总结

    传输网技术总结

    物理层技术

    骨干网

    • PSTN-公众交换电话网络
      就是最早的模拟信号的电话网,使用程控交换机,可以通过Modeam传输数字信号,速率只有56Kbit/s(实际还要打折扣),打电话时不能上网。
      1962年美国贝尔在发布了第一台Modem,可以在电话线上传输数字信号。
      最早的电话线路是直连的,后来出现了人工电话交换,机电式交换机,程控交换机。
      1965年美国贝尔生产世界上第一台程控电话交换机。
      最早的数字电话信号传输使用PCM编码的TDM技术,主要有E1(欧洲,1962年)和T1(美国,1962年)两种技术标准,T1,E1线路用于电话交换机之前的互联。
      1970年,法国阿尔卡特推出了第一台数字程控交换机E10,使用PCM/TDM中继传输信号。
      2017年中国电信最后一台TDM交换机下线,标志着PSTN在中国彻底退出历史舞台。
    • PDH-准同步数字体系
      PDH是数字通信技术,可以把多路电话调制到一条数字线路上进行传输,是早期构成数字电话网的基础。
      1972年ITU-T的前身CCITT提出了第一批PDH建议,1976年和1988年又提出了两批建议,对原有建议进行补充完善。PDH没有统一的标准,主要有E1,T1,J1三个标准
      E1 32个时隙,速率2.048Mbit/s,我们一般称为2兆线或者30B+D,是目前我们主要的接入方式,主要用户欧洲,中国。
      T1 24个时隙,速率1.544Mbit/s。T1系统主要应用在美国、加拿大、香港、台湾和日本(日本有些特殊,所以有厂家称之为J1)等地。
      E1通过复用可以实现2.048Mbps(E1),8.448Mbps(E2),34.368Mbps(E3),139.264Mbps(E4),565.148Mbps(E5)
      T1通过复用可以实现1.544Mbps(T1),6.312Mbps(T2,44.736Mbps(T3),274Mbps(T4)
      E1标准采用32个时隙,支持30路语音通话(另外两个时隙有别的用途)。T1标准采用24个时隙支持24路语音通话。
      PDH的传输介质最初是市话双绞电缆,随后发展到同辆电缆和微波,随着光纤的发明,后来也支持光纤。
      PDH可以承载PPP,HDLC,Frame-relay,ATM,GFP等协义
      PDH现已补淘汰
    • SDH-同步数字体系
      1988年,国际电信组织(ITU-T)通过了第一批SDH提义,提供了全球统一的标准,使用统一的时钟,通过时分复用来聚合不同的业务。主要以光纤为传输介质,也支持微波和卫星。
      SDH以STM-1为基本速率,主要的速率等级有STM-0(51.84Mbits/s),STM-1(155MBit/s),STM-4(622Mbit/s),STM-16(2.5Gbit/s),STM-64(10Gbit/s),STM256(40Gbit/s)。
      SDH可以承载PPP,HDLC,Frame-relay,ATM,MPLS,MSTP等协义。SDH不能直接传输IP数据,需要封装为PPP后再传输。
      1996年中国第一个骨干网chinanet投入运营,采用SDH技术,提供622Mbps的连接速率。
      SDH正在被OTN取代,预计2030年以后会完全过渡到OTN网络。
    • OTN-光传送网
      基于波分复用的光传输网,由ITU-T于1998年提出。OTN支持光交叉和电交叉(属于光/电路交换),光交叉通过光交换机实现,利用光交叉可以组建全光网。
      OTN的速率等级:ODU0(1.24Gbps对应以太网1G),ODU1(2.49Gbps),ODU2(10.03Gbps),ODU2e(10.39Gbps对应以太网10G),ODU3(40.3Gbps,40G以太网),ODU4(104Gbps,100G以太网),OTUC4(421Gbps,400G以太网)。
      OTN是一个分层的架构,可以映身不同速率和协议的数据,如PPP,帧中继,SONET/SDH,ATM,Ethernet,IP,PDH,FibreChannel,GFP,MPLS,OTN虚级联,ODU复用等。OTN已经跟电话网没有关系了,在OTN上传输的就是单纯的比特流。
      虽然OTN可以直接传输IP数据,但是通常会提供以太网接口。
      参考:
      https://en.wikipedia.org/wiki/Optical_transport_network

    城域网

    • M-OTN
      基于OTN的域域网,中国电信提出,主要引入了OSU(在ODU Flex的基础上支持无损速率调整 ),ODU25,ODU50。M-OTN应该主要用于固网政企客户。
    • STN
      是中国电信在5G时代用于移动回传、固网接入、边缘CDN以及应对MEC下沉的承载技术。是从4G时代的IPran技术演进而来。STN的L1层是FlexE,L2层是SRv6 EVPN。
    • PTN
      分组传送网,是以以太网(也可以是SDH,OTN,中国移动应该是以以太网为主)为基础结合MPLS-TP形成的分组传送网。
    • MTN
      是以分片以太网为基础,在4G/5G移动承载网场景发展出来的城域网技术。可以对以太网链路进行切片拆分为多个子通道。在接入层用灰光,汇聚层用彩光DWM。传输介质使用FlexE。MTN最低速率粒度是5Gbps,细粒度的fgMTN与fgOTN一致可以做到10Mbps。
    • 细粒度OTN
      在ODU上增加一个时分复用层
      OSUFlex/OSU:是中国厂商推出的方案,支持2.6Mbps的粒度划分速率等级,但没有完全基于TDM,时隙和时延不固定。
      fgOTN: 是ITU-T标准化的方案,支持10Mbps的粒度划分速率等级,完全基行TDM,时延固定。fgOTN标准于2024年正式发布。
    • 细粒度MTN
      fgMTN,是ITU-T标准化的方案,支持10Mbps的粒度划分速率等级,完全基行TDM,时延固定。fgOTN标准于2024年正式发布。
    • 固移融合城域网(新型城域网)
      以前的城域网是公网互联网,4G/5G移动承载网,云网(数据中心)分开建设(可能因为以前没有成熟的分片技术),这样建设成本高,不同网间互联延迟高。
      中国电信基于STN(FlexE+SRV6 EVPN)构建了融合固移云的新型城域网,固网,移动,和云网实现全互联无阻塞转发,减少了网络跳数,号称一跳入云。中国电信的新型城域网目前只有小规模的试点,于2024年5月16日发布的”智云上海”就是此类技术。
      中国联通的融合城域网叫智能型城域网,中国联通也是基于SRv6 EVPN技术在深圳,青岛等地有试点。
      中国移动还未见相关计划。

    分组交换技术

    X.25,帧中继,ATM,MPLS都属于分组交换技术,可以在各种物理层上提供虚电路服务。X.25和帧中继主要在电 话线上运行,MPLS主要在光纤线路上运行。

    • X.25
      X.25由ITU-T在1976年定义,以电话线或ISDN线路作为传输介质提供永久虚电路服务,在X.25上可以承载IP协议,但是也可以直接在X.25上传输应用数据(IP协议1981年才提出)。
      X.25是一个三层协议,涵盖了物理层,链路由,网络层。X.25本身是以分组传输数据的,但是可以向上层提供面向连接的服务。
      在X.25网络中当设备作为终端通信设备时称为DCE,DTE,作为交换机使用时称为PSE,PSE由运营商来部署。
    • 帧中继

      帧中继兴起于1992年,是在X.25的基础上改进而来,去掉了X.25的网络层和数据纠错功能,所以帧中继是一个二层协议。帧中继可以动态的调整每个虚电路的带宽,所以会存在带宽超卖的问题。
      帧中继可以在ISDN,PDH网络上运行。

    • ATM-异步传输模式
      1992年6月被ITU-T指定为B-ISDN的传输和交换模式。采固定的信元长度,数据交换实现起来更简单。与帧中继一样可以提供面向连接的分组交换服务,可以承载IP,PPP,以太网等多种上层协议。现在已被MPLS取代。
      ATM信元以固定长度,固定速率发送,没有数据要传输时则发送空闲信元(听着很耗电)。ATM交换机通过识别VPI/VCI标识进行数据交换,不同的标识代表了不同的虚连接。
    • MPLS-多协议标签交换

      MPLS是1998年由IETF提出的。
      ATM(Asynchronous Transfer Mode)技术采用定长标签(即信元),并且只需要维护比路由表规模小得多的标签表,能够提供比IP路由方式高得多的转发性能。然而,ATM协议相对复杂,且ATM网络部署成本高,这使ATM技术很难普及。传统的IP技术简单,且部署成本低。为了结合IP与ATM的优点,多协议标签交换技术MPLS(Multiprotocol Label Switching)就此产生。
      MPLS最初是为了提高设备的转发速度而提出的。与传统IP路由方式相比,它在数据转发时,只在网络边缘分析IP报文头,而不用在每一跳都分析IP报文头,节约了处理时间。
      随着ASIC技术的发展,路由查找速度已经不是阻碍网络发展的瓶颈。这使得MPLS在提高转发速度方面不再具备明显的优势。但是MPLS支持多层标签和转发平面面向连接的特性,使其在VPN(Virtual Private Network)、流量工程、QoS(Quality of Service)等方面得到广泛应用。
      MPLS使用一个20bit的标签,MPLS交换机根据标签值进行数据转发。MPLS的转发是依懒于路由表的,属于2.5层协议。
      MPLS只能承载IPv4,IPv6,IPX等三层协议,不像ATM还可以承载二层协议。

    • MPLS-TP/PTN/P-OTN/MS-OTN

      2008年IETF和ITU-T开始合作开发MPLS-TP,于2010开始陆续发布相关标准。
      MPLS-TP是MPLS去掉了对三层的依懒,同时增加了OAM功能,然后又融合了PWE3(L2VPN,可以仿真ATM、帧中继、以太网、低速TDM等,提供了承载二层业务的能力)。以太网(或FlexE)+MPLS-TP称为PTN,OTN+MPLS-TP称为MS-OTN或P-OTN。

    • OMPLS

      OMPLS是一项光分组交换技术,把MPLS标签映射到不同的波长上光交换机根据波长进行光信号交换。是一项还在发展中的技术,可以实现全光网络。目前主要使用的光路交换,光分组交换能不能成为主流还要看未来的发展。

    • SRv6

      基于IPv6的分段路由技术,可以用于骨干网的数据转发,这样就实现了IPv6技术一统天下,不需要在骨干网专门搞一套分组交换技术了,取代现在使用的MPLS,华为,中兴等厂商都在积极推进。不过SRv6只是刚出现的新技术,还没有大规模使用,将来能不能取代MPLS还不一定。

    其它协议

    • MSTP-多业务传送平台
      因为SDH是为电话网设计的,并不能承载各种数据业务,只能把其它业务封装为MSTP帧然后在SDH线路上传输。MSTP可以封装以太网,IP,TDM,ATM等。MSTP只是转换协议格式以方便在SDH上传输,不提供虚线路。
    • VPLS,VxLAN
      VPLS就是在MPLS网络上传输标准的以太网数据,把MPLS模拟成一个以太网交换机来转发以太网数据,VPLS是Mesh结构的网络,不同于二层VPN的点到点连接,VPLS报文在VSI节点间中继发转,就像帧中继一样。VPLS是一个在分组交换层上构建的overlay以太网。使用PBB报文封装的VPLS称为PBB VPLS。
      VxLAN则是在IP网络上上构建的Overlay以太网或者说虚拟以太网,VxLAN使用UDP报文封装。VXLAN多使用于云计算场景中,新型互联网交换中心也使用了VxLAN。不同于VPLS,VxLAN不会模拟以太网交换机功能,NVE(Network Virtualization Edge)并不会把VxLAN报文转发到下一个NVE,所以就要求同一BD下的NVE能通过IP点对点完全连接。
    • EVPN
      EVPN是把网络节点的MAC地址学习用BGP协议来实现,分离了控制面和转发面,更易于实现大规模的虚拟以太网。EVPN可以基于MPLS实现也可以基于SRv6实现。VPLS结合EVPN称为EVPN VPLS也简称为EVPN,VxLAN与EVPN结合称为EVPN VxLAN。
      基于MPLS的EVPN于2015年由IETF(RFC7432)发布,基于SRv6的EVPN于2023年由IETF(RFC9252)发布。

    总结:
    – ATM,MPLS-TP等分组交换技术提供的是虚电路服务,可以把一条物理线路模拟成多个不同的虚线路,其实就是二层VPN。如果虚拟的线路过多就会造成带宽超卖,网络质量不达标。
    – SDH,OTN数据帧是以因定长度,固定速率发送的,即使没有数据也会发送空数据。
    – 骨干网属于运营商组建的局域网,不支持不同网络间的互联,数据进入或离开骨干网时需要依懒IP协义进行互联,在同一骨干网内部使用分组交换或电路交换技术进行数据转发。
    – 新型互联网交换中心(IXP) 由中立机构建立的互联网交换中心,可以让企业与企业之间互相联接,不再仅限于运营商管道,也可以用BGP方式与运营商联接,IXP只收取很低的端口中占用率。目前已有上海,深圳,杭州,中卫4个IXP,北京在建中。

    Views: 3

  • 华为AR系列路由器PPPOE拨号上网

    • 创建dialer接口
    acl 3998
     rule 10 permit ip
    interface Dialer1
     link-protocol ppp
     ppp ipcp default-route
     ppp chap user pppoe-user
     ppp chap password cipher 123456
     ppp pap local-user pppoe-user password cipher 123456
     ppp ipcp dns admit-any
     ppp ipcp dns request
     ipv6 enable
     tcp adjust-mss 1452
     ip address ppp-negotiate
     dialer user pppoe-user
     dialer bundle 1
     dialer number 1 autodial
     ipv6 address auto link-local
     ipv6 address auto global default
     ipv6 mtu 1492
     nat outbound 3998
     dhcpv6 client pd v6pd1
     sa application-statistic enable
    
    • 绑定pppoe会话到wan 0/0/9端口
    interface GigabitEthernet0/0/9
     pppoe-client dial-bundle-number 1
     ip address 192.168.1.2 255.255.255.0
     nat outbound 3997
     ip accounting input-packets
     ip accounting output-packets
     sa application-statistic enable
    
    • 配置lan口
    dhcpv6 pool pool1
     dns-server FD11::1
    interface Vlanif101
     mtu 1492
     ipv6 enable
     ip address 192.168.101.1 255.255.255.0
     ipv6 address v6pd1 ::1:0:0:0:1/64
     ipv6 address FD11::1/64
     ipv6 address auto link-local
     ipv6 address auto global
     ipv6 mtu 1492
     undo ipv6 nd ra halt
     ipv6 nd autoconfig other-flag
     dhcp select interface
     dhcpv6 server pool1
     dhcp server dns-list 192.168.101.1
    

    注意:
    如果ipv6不稳定可以添加以下配置:
    tcp ipv6 max-mss 1432
    在lan口上配置mtu是因为路由器的path mtu不能正常起作用,正常情况下是不需要的。

    Views: 12

  • 华为USG系列防火墙PPPOE拨号上网

    • 创建dialer接口
    interface Dialer0
     link-protocol ppp
     ppp chap user pppoe-user
     ppp chap password cipher 123456
     ppp pap local-user pppoe-user password cipher 123456
     ppp ipcp dns admit-any
     mtu 1492
     ipv6 enable
     ip address ppp-negotiate
     dialer user pppoe-user
     dialer bundle 1
     dialer number 0 autodial
     ipv6 address auto link-local
     ipv6 address auto global
     ipv6 mtu 1492
     healthcheck wan0
     service-manage https permit
     service-manage ssh permit
     redirect-reverse
     ipv6 redirect-reverse
     dhcpv6 client pd v6pd1
    
    • 创建健康检查
    healthcheck name wan0
    destination 223.5.5.5 interface Dialer0  protocol dns
    
    • 创建NAT
    nat-policy
     rule name Internet
      source-zone 办公网
      egress-interface Dialer0
      action source-nat easy-ip
    
    • 创建默认路由
    ip route-static 0.0.0.0 0 Dialer0
    ipv6 route-static :: 0 Dialer0
    
    • 绑定pppoe会话到wan0口
    interface WAN0/0/0
     pppoe-client dial-bundle-number 1
     undo shutdown
     anti-ddos flow-statistic enable
     service-manage https permit
     service-manage ssh permit
    
    • 配置lan口
    dhcpv6 pool pool1
     dns-server FD06::1
    interface Vlanif6
     ipv6 enable
     ip address 192.168.2.1 255.255.255.0
     ipv6 address v6pd1 ::1:0:0:0:1/64
     ipv6 address FD06::1/64
     ipv6 address auto link-local
     ipv6 address auto global
     ipv6 mtu 1492
     undo ipv6 nd ra halt
     ipv6 nd autoconfig other-flag
     alias Vlanif6
     service-manage http permit
     service-manage https permit
     service-manage ping permit
     service-manage ssh permit
     dhcp server mask 255.255.255.0
     dhcp server ip-range 192.168.2.1 192.168.2.254
     dhcp select interface
     dhcpv6 server pool1
     dhcp server gateway-list 192.168.2.1
     dhcp server dns-list 192.168.2.1
    

    注意:
    如果拨号成功后ipv6不通,可以删除并重新配置以下命令:

     ipv6 address auto link-local
     ipv6 address auto global
     dhcpv6 client pd v6pd1
    

    如果ipv6上网不稳定还可以配置以下命令:

    tcp ipv6 max-mss 1432
    

    Views: 11

  • debian系统自动签名dkms模块

    • 启动自动签名
      打开/etc/dkms/framework.conf,反注释以下两行
      >mok_signing_key=/var/lib/dkms/mok.key
      mok_certificate=/var/lib/dkms/mok.pub
    • 导入mok到主板
      mokutil --import /var/lib/dkms/mok.pub
      这条命令执行后会提示输入密码,重启动需要在Bios界面输入相同的密码进行确认才能启动

    Views: 15

  • ubuntu从传统模式切换到uefi安全启动

    • 创建一个大小为100-200MB的FAT32分区
      可以用fdisk,parted,gparted等分区工具创建,分区类型为主分区,开启boot标志。如果没有剩余空间可以用livecd启动调整分区大小
    • 安装uefi启动相关依懒
      apt install shim-signed grub-efi grub-efi-amd64-signed
      shim-signed 是使用微软签名的uefi启动文件,是支持安全启动必须的
      grub-efi 为grub启用efi支持,但是不支持安全启动
    • 修复uefi启动
      ubuntu有三种方式可以修复启动

      1. 纯手工
        • 用ubuntu livecd启动进行桌面
        • 执行以下命令
          mount -t efivarfs none /sys/firmware/efi/efivars
          mount /dev/sdb3 /mnt
          mount /dev/sdb2 /mnt/boot
          mount /dev/sdb1 /mnt/boot/efi
          mount /dev/sdb2 /mnt/
          mount /dev/sdb1 /mnt/boot/efi
          for i in /dev /dev/pts /proc /sys /sys/firmware/efi/efivars /run; do mount -B $i /mnt/$i; done
          chroot /mnt
          grub-install /dev/sdb
          update-grub
          

          按Ctrl-D退出chroot
          重启电脑或虚拟机

      2. boot-repair包
        • 安装boot-repair
          sudo add-apt-repository ppa:yannubuntu/boot-repair && sudo apt update
          sudo apt install -y boot-repair && boot-repair
          
        • 修复
          启动boot-repair,点击Recommended repair或者点开高级选项,然后分别选择efi分区,boot分区,系统分区,最后点击apply按钮
      3. 使用boot-repair disk
        boot-repair disk实际上就是一个集成了bot-repair的livecd,相当于Windows上的WinPE

        • 下载boot-repair disk
          https://sourceforge.net/projects/boot-repair-cd/files/
        • 使用下载的iso启动系统
          下面的步聚跟liveCD方式一样了

    注意:
    使用iso启动时若不能启动可以暂时关闭bios中的安全启动
    以上方法即可以用于轨换启动式到uefi,也可以用于修复uefi引导
    我是先在bios中切换到uefi启动然后去修复操作系统的引导的,不知道先修复引导再改bios行不行

    Views: 9

  • IKEv2对Ed25519,Ed448的支持

    1. 数字签名认证
      IANA(https://www.iana.org/assignments/ikev2-parameters/ikev2-parameters.xhtml)
      定义的IKEv2认证方法中并不包括EdDSA,因为椭圆曲线有很多种,每一种曲线都定义一种认证方法会难为管理,所以在RFC7427中定义了一个名为Digital Signature的数字签名认证方法,可以支持所有的签名算法,具体用哪种算法则由ASN.1 OID来区分。
      ed25519算法的OID是1.3.101.112(RFC8410定义)
      RFC8420定义了EdDSA的具体要求,RFC8420发布于2018年,所以可以说IKEv2在2018年引入了ed25519或ed448数字签名认证的支持。

    参考:
    https://docs.strongswan.org/docs/5.9/pki/pkiQuickstart.html

    1. 密钥协商
      RFC 8031定义了DH组31和32,分别代表x25519,x448算法。strongswan也提供了dh组31和dh组32的支持。
      参考:
      https://wiki.strongswan.org/projects/strongswan/wiki/552

    Curve25519和Curve448曲线的定义:
    https://datatracker.ietf.org/doc/rfc7748/

    2018年发布的TLS1.3也支持Ed25519和Ed448签名算法

    Views: 31

  • 加密算法简介(含国密)

    国际算法

    • DES 旧的对称加密算法,已废弃
    • AES 新的对称加密算法,用于取代DES
    • AES-GCM 加密的同时提供消息验证,密文和验证码(MAC)一起发送,也就是AEAD,其它不带GCM后缀的AES算法不支持AEAD,ipsec和tls1.2,1.3都支持gcm模式
    • Blowfish 对称加密,已废弃
    • chacha20 流式对称加密,使用256位密钥长,对应AES256,安全性与AES相当或略优于AES,纯软件件计算比AES快3倍,比硬件AES慢1倍。这个算法以后用的会越来越多,在没有AES指令集的设备上chacha20是首选算法。由RFC 8439定义。
    • chacha20-poly1305 chacha20的AEAD版
    • RC4 对称加密,特点是速度非常快,比硬件加速的AES还快,已废弃
    • RSA 非对称加密算法,可用于消息加密,消息签名,密钥交换(低于1024位不安全)
    • ECC 非对称加密算法,另外用于密钥交换时用ECDH,用于消息签名时用ECDSA,一般不直接用于加密,根据不同的密钥长度分为P-192, P-224, P-256, P-384 和 P-521。
    • curve25519 非对称加密算法,蒙哥马利曲线,另一种椭圆曲线,256位的密钥长度,比ECC速度更快,更安全
    • curve448 非对称加密算法,448位的密钥长度,curve25519的加强版,安全性更高,但速度没有curve25519快
    • DSA 非对称加密算法,只能用于消息签名,但是用于签名时相比RSA也没有啥优势,所以使用并不多
    • ECDSA DSA的ECC版,ECC一般不直接用于签名,目前基于ECC的数字签名算法就是ECDSA
    • EDDSA 基于爱德华兹曲线的签名算法,速度更快,更安全,用于取代ECDSA,因为NIST有往算法里掺沙子的前科。
    • ED25519 是EDDSA的curve25519实现,密钥长度是256位
    • ED448 是EDDSA的curve448实现,密钥长度448位
    • DH 非对称加密算法,只能用于密钥交换
    • ECDH DH的ECC版,ipsec中DH group 19,20,21使用的是ECDH,其它使用的是DH
    • X25519,X448 ED版的DH算法,分别对应curve25519,curve448,ipsec中对应dh31和dh32
    • DHE,ECDHE 使用临时公私钥的DH,ECDHE算法,提供PFS能力,TLS1.3上PFS是必选的,IPsec可选支持PFS
    • MD5 散列算法,128位,速度很快,已废弃
    • SHA-1 散列算法,160位,已废弃
    • SHA-2(SHA256-SHA512) 散列算法
    • SHA-3(SHA3-256-SHA3-512) 散列算法,长度与sha2一致但使用了不同的算法,是SHA2的备选。目前并没有消息说SHA3比SHA2更安全,只是没有与SHA2相同的弱点。
    • HMAC 加了密码的散列算法,即可以用于散列,也可以用于身份验证(密码不一样,散列结果不一样),HMAC可以与任意其它散列算法一起使用,这样散列算法也实现了AEAD

    国密算法:

    • SM1 相当于AES128,算法不公开,用于小数据量的加解密
    • SM2 基于ECC,相当于ECC256,ECDH,ECDSA,可用于加密,签名,密钥交换,USG6525E防火墙支持使用sm2进行身份认证
    • SM3 相当于SHA-256,但比sha-256g速度慢,支持MAC的版本为HMAC-SM3,USG6525E防火墙不支持IKEV2的完整性验证,但支持sm3用于IKEV1的完整性验证
    • SM4 与SM1一样都是对称加密,用于大数据量的加解密,也可以支持GCM模式(AEAD),USG6525E防火墙中可用于IPsec的数据加密,但不支持GCM模式,所以还需要另外配置完整性验证(使用GCM模式时esp authentication-algorithm不再生效)
    • SM7 与SM1一样,都是128位密钥和分组,主要用于非接触式IC卡,算法不公开
    • SSF33 对称加密,使用较少,算法不公开
    • SM9 非对称加密,使用用户标识(手机号,姓名,email地址)作为公钥,免去了证书创建,使用更方便,属于轻量级的非对称加密
    • ZUC 流式对称加密,可支持AEAD,已成为LTE标准

    注意:
    ECC256的安全性相当于RSA3072
    ECC256的破解难度相当于AES128,对称密码的破解难度要比非对称密码高
    SECP定义的曲线以r1作为后缀的与NIST定义的曲线是相同的,比如secp256r1等于nist p-256,secp38441等于nist p-384,这个定义secp曲线的seca组织是NIST的马甲吗?
    这里可以查看各个曲线的参数定义
    https://neuromancer.sk/std/secg/secp256r1
    secp256k1不是由nist定义的,比特币用的也是这个算法
    brainpool系列曲线是由rfc5639定义的,比nist的更安全,但是速度要慢得多,所以一般用ed25519的多
    爱德华兹曲线相关的ed25519,x25519,ed448,x448是由IETF定义的
    TLS1.3提供了对cure25519,cure448,AEAD的支持,USG6525E支持AEAD但不支持cure25519和cure448
    sha1,sha2,sha3,dsa,ecdsa,aes,ecc NIST P-xxx,des,hmac等算法都是由nist定义的
    关于ipsec中的dh组应该是由ietf自己定义的,可以参考RFC6071,国密算法中没有定义dh组
    各种椭圆曲线算法中只有国密的SM2是能直接加解密数据的,NIST,SECG,IETF定义的ECC和ED25519都不支持加解密数据,而只能用于签名和密钥交换
    2012年起中国国家密码局陆续发布国密算法,2013发生棱境门事件并爆出了Dual_EC_DRBG后门
    中国国家密码局2014年发布GM/T 0022-2014 IPSec VPN技术规范,2023年发布了GM/T 0022-2023 IPSec VPN技术规范
    中国国家密码局发布的商秘(SM)算法不能用于处理国家秘密,只能用于处理商业秘密
    AEAD把认证数据分为加密数据和非加密数据(关联数据),通过一个MAC保证加密部分和非加密部分均未篡改。通常IP数据包内容是需要加密的数据,IP数据包头是不需要加密的信息。

    Views: 143

  • 上海移动IPTV单线复用

    1. 运营商光猫桥接
      拓朴图:

      这是最简单的一种实现方法,拓扑图是最简化的情况,交换机可以多个串联,还可以使用支持Vlan的交换机。
      pon stick设置

      其中关键的几点:
      连接模式 桥接
      IP模式 IPv4&IPv6
      DHCP Server关闭
      MTU 1500
      启用Vlan 选中
      Vlanid 1101
      LAN端口绑定 LAN1

      其中关键的几点:
      连接模式 桥接
      IP模式 IPv4&IPv6
      MTU 1500
      启用Vlan 选中
      Vlanid 1103
      Openwrt设置
      IPv6 ULA 前缀:设置为空,机顶盒不支持ula并会导致机顶盒ipv6不通,如果不需要机顶盒的ipv6支持也可以不设置为空
      防火墙设置

      网络拓扑
      如果使用的是管理型交换机自己根据需要配置,至于网上提到的dhcp option125我这里没有用到,如果有用到的可以参考网上的贴子。我这里把vlan配置好就能用了。
      openwrt中dhcp option可以这样配置(不必要):
    2. 猫棒+openwrt路由器
      我用的是MA5671A,在猫棒上把互操作兼容模式关闭,这样把所有的Vlan透传到路由器
      网络拓朴

      openwrt上把所有端口放到一个网桥中,把openwrt路由器当成一个交换机然后划分vlan

      重点是其中的1101,1103,50这3个vlan,这是从pon网络传过来的。31,33这两个vlan是在本地创建的,我把局域网和iptv的网段通过vlan分隔了。其它的vlan没有用,我只是把pon网络支持的vlan全配置了。
      防火墙的配置参考上面,IPv6 ULA 前缀也与上面一致。
      拨号的时候绑定vlan 1101

      iptv内网绑定vlan 31,lan绑定vlan 33
      这样做理论上是没有问题的,但是我每次打开机顶盒都要重启路由器,不然就会像看幻灯片一样,抓包发现会有大量丢包,而且vlan 1103的tag标签时不时的会丢掉。vlan标签丢掉是造成丢包的一个原因,是不是所有原因还不清楚。基本上可以确定vlan标签在网桥中透传时出错了。还不确定是个例还是openwrt的通病。
    3. pon stick插到交换机上
      拓朴图

      这种方法我没有试,但可以绕过openwrt用交换机进行vlan透传,避免openwrt造成的丢包问题。

    注意:
    MTU正确设置也是很重要的,ppoe连接上一定要打开mss钳制,连接的mtu也要设置正确。
    vlan说明:1101是pppoe,1103是单拨iptv,50是组播,110是tr069,1102是voip,其中有用的是1101和1103。lan端口绑表表示untag口,否者就是tag口
    我在配置的过程中影响比较大的还是vlan和mtu,这两点没问题就能通。另外机顶盒对ipv6兼容性不是很好也是要注意的地方。
    有线和无线尽量不放在一个vlan或网段中,因为有线网络中的广播以及组播包会干扰无线网络。

    Views: 122

  • pandas的index

    • 定义index
      1. 读取csv时
        data = pd.read_csv('data.csv', index_col='id')
        这种方法会创建命名的索引列,并且会放在第1列(不管csv文件中是第几列)
      2. 声明DataFrame时
        data = [['Google', 10], ['Runoob', 12], ['Wiki', 13], ['Baidu', 20], ['taobao', 15]]
        # 创建DataFrame
        df = pd.DataFrame(data, columns=['Site', 'Age'], index=[1,2,3,4,6])
        

        这种方法会创建没有名字的索引列,仍然放在第1列。

    • 遍历时使用index
      1. 使有iterrows()
        for index, item in df.iterrows():
        print(index, item) #index的值就是上述定义的index值,并不是连续的序号
        print(item[0]) #item[0]就是第1列Site的值,item中不包含index列
        
      2. 使用itertuples()
        for item in df.itertuples():
        print(item)
        print(item.Index) # i.Index就是索引列的值
        print(item[0]) # 与item.Index的值相同,也是id列的值,id列会强制移动第0列。
        print(item[1]) # Site列的值
        

        如果原始数据中没有index会自动生成一个递增的index列。

      3. 无index的itertuples()
        for item in df.itertuples(index=False):
        print(item)
        #print(item.Index) # 报错
        print(item[0]) # Site列的值
        print(item[1]) # Age列的址
        

        id列或者index列会被从Series中移除,无法访问id列。

    Views: 0

  • Artificial Intelligence vs Machine Learning vs. Deep Learning

    Machine Learning vs Artificial Intelligence: Whats the Difference?

    ml and ai meaning
    The existence of current AI/ML capabilities does not mean a private equity firm will not have to invest significantly in improving AI/ML, particularly if the training datasets will need to be overhauled post-close. Developed by OpenAI, GPT-4 is one of the largest publicly available LLM models. It has been trained on a large amount of data and has higher accuracy and ability to generate text than previous models.

    Yet, their intricate interplay and unique characteristics often spark confusion. In this article, we embark on a journey to demystify the trio, exploring the fundamental differences and symbiotic relationships between ML vs DL vs AI. Unravel the intricacies of each domain and gain a comprehensive understanding of how these transformative technologies collectively shape the future of intelligent systems and drive unparalleled advancements in our digital landscape.

    ml and ai meaning

    An alternative is to discover such features or representations through examination, without relying on explicit algorithms. They let the machines learn independently, ingesting https://chat.openai.com/ vast amounts of labeled data and unlabeled data to detect patterns. Advancements in big data and the vast data we have collected enabled machine learning in the first place.

    Learning in ML refers to a machine’s ability to learn based on data and an ML algorithm’s ability to train a model, evaluate its performance or accuracy, and then make predictions. This subcategory of AI uses algorithms to automatically learn insights and recognize patterns from data, applying that learning to make increasingly better decisions. Reinforcement learning is the most complex of these three algorithms in that there is no data set provided to train the machine. Instead, the agent learns by interacting with the environment in which it is placed. It receives positive or negative rewards based on the actions it takes, and improves over time by refining its responses to maximize positive rewards.

    What is artificial intelligence (AI)?

    As AI/ML continues to grow in value and capability, consistent leading practices for compliance and data management must factor into growth plans through an end-to-end AI/ML due diligence framework. In light of anticipated changes in legal and compliance regulations, private equity firms should adopt a rigorous end-to-end assessment as a key best practice to ensure they remain in compliance with the new requirements. The relative “newness” of AI/ML for most private equity firms means there is a lot of confirmation bias around AI/ML capabilities.

    A. AI and ML are interconnected, with AI being the broader field and ML being a subset. Through integrating the Epicor Catalog–a comprehensive, cloud-based database with access to over 17 million SKUs from 9,500+ manufacturers– Carvana has dramatically increased productivity and cut the cost per unit for parts by more than 50%. Many companies have successfully integrated Epicor’s AI and ML solutions for a remarkable transformation in their business operations. Despite their prevalence in everyday activities, these two distinct technologies are often misunderstood and many people use these terms interchangeably.

    Large language models serve as foundation models, providing a basis for a wide range of natural language processing (NLP) tasks. Generative AI can encompass a range of tasks beyond language generation, including image and video generation, music composition, and more. Large language models, as one specific application of generative AI, are specifically designed for tasks revolving around natural language generation and comprehension.

    ml and ai meaning

    So now you have a basic idea of what machine learning is, how is it different to that of AI? We spoke to Intel’s Nidhi Chappell, head of machine learning to clear this up. For example, suppose you were searching for ‘WIRED’ on Google but accidentally typed ‘Wored’. After the search, you’d probably realise you typed it wrong and you’d go back and search for ‘WIRED’ a couple of seconds later. Google’s algorithm recognises that you searched for something a couple of seconds after searching something else, and it keeps this in mind for future users who make a similar typing mistake.

    Specific practical applications of AI include modern web search engines, personal assistant programs that understand spoken language, self-driving vehicles and recommendation engines, such as those used by Spotify and Netflix. The system used reinforcement learning to learn when to attempt an answer (or question, as it were), which square to select on the board, and how much to wager—especially on daily doubles. Explore the benefits of generative AI and ML and learn how to confidently incorporate these technologies into your business. Ensure that team members can easily share knowledge and resources to establish consistent workflows and best practices. For example, implement tools for collaboration, version control and project management, such as Git and Jira. In its most complex form, the AI would traverse several decision branches and find the one with the best results.

    Further Differences Between AI and Machine Learning

    In this blog post, we may have used or referred to third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.

    AWS offers a wide range of services to help you build, run, and integrate artificial intelligence and machine learning (AI/ML) solutions of any size, complexity, or use case. To paraphrase Andrew Ng, the chief scientist of China’s major search engine Baidu, co-founder of Coursera, and one of the leaders of the Google Brain Project, if a deep learning algorithm is a rocket engine, data is the fuel. Unlike machine learning, deep learning uses a multi-layered structure of algorithms called the neural network.

    Even though we talked about machine learning being more limited in scope, it does make it possible for AI tools to solve and address varied problems across different sectors. Machine learning is behind many of these applications, making it possible for AI to be so dynamic. For AI, you can use AWS services to build your own AI solutions from scratch or integrate prebuilt artificial intelligence (AI) services into your solution. ML is best for identifying patterns in large sets of data to solve specific problems.

    But while AI and machine learning are very much related, they are not quite the same thing. The intention of ML is to enable machines to learn by themselves using data and finally make accurate predictions. We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

    ml and ai meaning

    Machine learning also incorporates classical algorithms for various kinds of tasks such as clustering, regression or classification. The more data you provide for your algorithm, the better your model and desired outcome gets. Machine learning is a relatively old field and incorporates methods and algorithms that have been Chat GPT around for dozens of years, some of them since the 1960s. These classic algorithms include the Naïve Bayes classifier and support vector machines, both of which are often used in data classification. In addition to classification, there are also cluster analysis algorithms such as K-means and tree-based clustering.

    Getting started in AI and machine learning

    This stems from the technology using existing content to inform how it creates its own “original” content. As the AI field continues to grow, questions will continue to be asked about its ethics, and it will be a challenge in its own right to decide on and enforce ways to keep everyone safe. You can infer relevant conclusions to drive strategy by correctly applying and evaluating observed experiences using machine learning. While we are not in the era of strong AI just yet—the point in time when AI exhibits consciousness, intelligence, emotions, and self-awareness—we are getting close to when AI could mimic human behaviors soon. We can compare the model’s prediction with the ground truth value and adjust the parameters of the model so next time the error between these two values is smaller.

    While ML is a powerful tool for solving problems, improving business operations and automating tasks, it’s also complex and resource-intensive, requiring deep expertise and significant data and infrastructure. You can foun additiona information about ai customer service and artificial intelligence and NLP. Choosing the right algorithm for a task calls for a strong grasp of mathematics and statistics. Training ML algorithms often demands large amounts of high-quality data to produce accurate results. The results themselves, particularly those from complex algorithms such as deep neural networks, can be difficult to understand. Semi-supervised anomaly detection techniques construct a model representing normal behavior from a given normal training data set and then test the likelihood of a test instance to be generated by the model.

    Though used interchangeably, here’s the real difference between artificial intelligence vs. machine learning vs. deep learning. Artificial intelligence performs tasks that require human intelligence such as thinking, reasoning, learning from experience, and most importantly, making its own decisions. As our article on deep learning explains, deep learning is a subset of machine learning.

    ml and ai meaning

    Many algorithms and techniques aren’t limited to a single type of ML; they can be adapted to multiple types depending on the problem and data set. For instance, deep learning algorithms such as convolutional and recurrent neural networks are used in supervised, unsupervised and reinforcement learning tasks, based on the specific problem and data availability. Machine Learning and Artificial Intelligence are two closely related but distinct fields within the broader field of computer science. It involves the development of algorithms and systems that can reason, learn, and make decisions based on input data.

    Machine learning vs. deep learning neural networks

    Similarly, decision-making and predictions are both key parts of nearly all AI tools. This is because assessing information, weighing up options, and deciding the best next step is an integral part of any intelligence. The machine learning algorithms analyze huge amounts of data to identify the patterns that facilitate this decision-making. AI’s primary goal is to mimic human intelligence and abilities, such as reasoning, decision-making, and adaptability. It achieves this with a combination of techniques, but the most critical method is almost always machine learning.

    That’s because these machine learning algorithms make it possible for the AI to analyze information, identify patterns, and adapt its behavior. Artificial intelligence (AI) is an umbrella term for different strategies and techniques you can use to make machines more humanlike. AI includes everything from smart assistants like Alexa to robotic vacuum cleaners and self-driving cars.

    • So now you have a basic idea of what machine learning is, how is it different to that of AI?
    • Artificial intelligence (AI) describes a machine’s ability to mimic human cognitive functions, such as learning, reasoning and problem solving.
    • This is because assessing information, weighing up options, and deciding the best next step is an integral part of any intelligence.
    • The problem is that these situations all required a certain level of control.

    The broader aim of AI is to create applications and machines that can simulate human intelligence to perform tasks, whereas machine learning focuses on the ability to learn from existing data using algorithms as part of the wider AI goal. Today, artificial intelligence is at the heart of many technologies we use, including smart devices and voice assistants such as Siri on Apple devices. In simplest terms, AI is computer software that mimics the ways that humans think in order to perform complex tasks, such as analyzing, reasoning, and learning. Machine learning, meanwhile, is a subset of AI that uses algorithms trained on data to produce models that can perform such complex tasks. DL is able to do this through the layered algorithms that together make up what’s referred to as an artificial neural network. These are inspired by the neural networks of the human brain, but obviously fall far short of achieving that level of sophistication.

    Unsupervised machine learning

    We define weak AI by its ability to complete a specific task, like winning a chess game or identifying a particular individual in a series of photos. Natural language processing and computer vision, which let companies automate tasks and underpin chatbots and virtual assistants such as Siri and Alexa, are examples of ANI. Many people use machine learning and artificial intelligence interchangeably, but the terms have meaningful differences. As the quantity of data financial institutions have to deal with continues to grow, the capabilities of machine learning are expected to make fraud detection models more robust, and to help optimize bank service processing. As outlined above, there are four types of AI, including two that are purely theoretical at this point.

    Implementing quality management systems to close the AI translation gap and facilitate safe, ethical, and effective health AI solutions npj Digital Medicine – Nature.com

    Implementing quality management systems to close the AI translation gap and facilitate safe, ethical, and effective health AI solutions npj Digital Medicine.

    Posted: Sat, 25 Nov 2023 08:00:00 GMT [source]

    For example, you can train a system with supervised machine learning algorithms such as Random Forest and Decision Trees. Data management is more than merely building the models that you use for your business. You need a place to store your data and mechanisms for cleaning it and controlling for bias before you can start building anything.

    Principal component analysis (PCA) and singular value decomposition (SVD) are two common approaches for this. Other algorithms used in unsupervised learning include neural networks, k-means clustering, and probabilistic clustering methods. Machine learning is a form of artificial intelligence (AI) that can adapt to a wide range of inputs, including large data sets and human instruction. The algorithms also adapt in response to new data and experiences to improve over time.

    In the real world, the terms framework and library are often used somewhat interchangeably. But strictly speaking, a framework is a comprehensive environment with high-level tools and resources for building and managing ML applications, whereas a library is a collection of reusable code for particular ML tasks. Reinforcement learning involves programming an algorithm with a distinct goal and a set of rules to follow in achieving that goal. The algorithm seeks positive rewards for performing actions that move it closer to its goal and avoids punishments for performing actions that move it further from the goal.

    While artificial intelligence (AI), machine learning (ML), deep learning and neural networks are related technologies, the terms are often used interchangeably, which frequently leads to confusion about their differences. Deep learning is a subset of machine learning that uses several layers within neural networks to do some of the most complex ML tasks without any human intervention. As with the different types of AI, these different types of machine learning cover a range of complexity. And while there are several other types of machine learning algorithms, most are a combination of—or based on—these primary three. Toloka is a European company based in Amsterdam, the Netherlands that provides data for Generative AI development. We are the trusted data partner for all stages of AI development from training to evaluation.

    The Meaning of Explainability for AI – Towards Data Science

    The Meaning of Explainability for AI.

    Posted: Mon, 03 Jun 2024 07:00:00 GMT [source]

    Oracle Cloud Infrastructure (OCI) provides the foundation for cloud-based data management powered by AI and ML. The field of AI encompasses a variety of methods used to solve diverse problems. These methods include genetic algorithms, neural networks, deep learning, search algorithms, rule-based systems, and machine learning itself. Inspired by DevOps and GitOps principles, MLOps seeks to establish a continuous evolution for integrating ML models into software development processes. By adopting MLOps, data scientists, engineers and IT teams can synchronously ensure that machine learning models stay accurate and up to date by streamlining the iterative training loop.

    Developers filled out the knowledge base with facts, and the inference engine then queried those facts to get results. Reinforcement learning is often used to create algorithms that must effectively make sequences of decisions or actions to achieve their aims, such as playing a game or summarizing an entire text. In this article, you’ll learn more about what machine learning is, including how it works, different types of it, and how ml and ai meaning it’s actually used in the real world. We’ll take a look at the benefits and dangers that machine learning poses, and in the end, you’ll find some cost-effective, flexible courses that can help you learn even more about machine learning. But still, there lack datasets with a great density that be used for testing AI algorithms. For instance, the standard dataset used for testing the AI-based recommendation system is 97% sparse.

    Views: 1