发表更新fedfab29 分钟读完 (大约4327个字)

fedml-4.与fabric通信

  • 直接采用“偷梁换柱”的模式,把修改后的代码复制到pip安装的位置

pip install的位置

通过python -m site命令查找包的安装路径

1
2
3
4
5
6
7
8
9
10
11
12
sys.path = [
    '/usr/share/python3',
    '/usr/lib/python310.zip',
    '/usr/lib/python3.10',
    '/usr/lib/python3.10/lib-dynload',
    '/home/tt/.local/lib/python3.10/site-packages',
    '/usr/local/lib/python3.10/dist-packages',
    '/usr/lib/python3/dist-packages',
]
USER_BASE: '/home/tt/.local' (exists)
USER_SITE: '/home/tt/.local/lib/python3.10/site-packages' (exists)
ENABLE_USER_SITE: True

思考：需要修改哪些代码？

  • 修改Aggregator。Aggregator的作用是（1）保存各个节点上传的本地模型；（2）对本地模型进行aggregate操作。
    • 分离aggregator的功能,分为本地Aggregator和链上Aggregator,本地Aggregator不保存模型,将收到的模型转发给区块链,聚集操作时先向区块链取模型,再进行聚集操作
  • 修改FedMLServerManager
    • 在适当位置调用http接口,适当根据逻辑需要修改其它代码
  • 修改ClientMasterManager
    • 在适当位置调用http接口,适当根据逻辑需要修改其它代码
  • 修改message_define,文件中定义了C/S之间相互通信的名称,参数名称
    • 对其适当增删,达到C/S间协同的目的
阅读更多
发表更新fedfab14 分钟读完 (大约2034个字)

fedml-3.Runner源码阅读

wandb 的使用

配置config.yaml

  • enable - true
  • wandb key
  • priject name
1
2
3
4
5
6
tracking_args:
  log_file_dir: ./log
  enable_wandb: true #enable
  wandb_key: e3be1b9a8ab45f14a6ff454009bc7ca07b8792ba #key
  wandb_project: fedml_mnist_test #project name
  wandb_name: fedml_torch_fedavg_mnist_lr

运行

阅读更多
发表更新fedfab5 分钟读完 (大约683个字)

fedml-2.Docker多容器配置

Docker安装

1
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
  • 中间遇到输出，提示建议使用for windows
1
2
WSL DETECTED: We recommend using Docker Desktop for Windows.
Please get Docker Desktop from https://www.docker.com/products/docker-desktop

创建容器并配置环境

验证是否可以使用gpu

阅读更多
发表更新fedfab3 分钟读完 (大约460个字)

fabric-9.服务器配置

centos yum错误

1
错误：为 repo 'appstream' 下载元数据失败 : Cannot prepare internal mirrorlist: No URLs in mirrorlist
1
2
sudo sed -i -e "s|mirrorlist=|#mirrorlist=|g" /etc/yum.repos.d/CentOS-*
sudo sed -i -e "s|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g" /etc/yum.repos.d/CentOS-*

查看显卡

1
lspci | grep -i vga
阅读更多
发表更新fedfab3 分钟读完 (大约387个字)

fabric-5.论文

需要实现

  1. 论文中的新的归一化算法
  2. 防止恶意模型更新
  3. 客户使用私钥签名模型
  4. fedml的节点作为mec服务器起训练作用，再对原始数据进行处理，模拟移动设备对数据添加噪声
  5. 智能合约中记录用户的有效交易，用于用户获取奖励
  6. 如何控制fabric的出块？
  7. 实现IPFS
  8. peer节点和fedml训练节点之间如何获取对方的公钥，如何避免中间人攻击

笔记

IPFS is a peer-to-peer distributed file system that enables distributed computing devices to connect with the same file system.

在IPFS上进行存储，区块链上存储Hash指针，用于找到文件（块大小限制）

在神经网络的中间层添加噪声(ε, δ)-differential，cnn全连接层作为噪声提取器

阅读更多
发表更新fedfab3 分钟读完 (大约502个字)

fabric-4.创建ca

官方教程

创建ca节点

第一次启动CA时，它查找fabric-ca-server-config.yaml文件，其中包含CA配置参数。

编写创建ca的docker-compose文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
version: '3.7'

networks:
  fed_fab:
    name: fabric_fedml

services:
  ca_org1:
    image: hyperledger/fabric-ca:latest
    labels:
      service: hyperledger-fabric
    environment:
      - FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
      - FABRIC_CA_SERVER_CA_NAME=ca-org1
      - FABRIC_CA_SERVER_TLS_ENABLED=true
      - FABRIC_CA_SERVER_PORT=8050
      - FABRIC_CA_SERVER_OPERATIONS_LISTENADDRESS=0.0.0.0:18050
    ports:
      - "8050:8050"
      - "18050:18050"
    command: sh -c 'fabric-ca-server start -b admin:adminpw -d'
    volumes:
      - ../crypto-config/fabric-ca/org1:/etc/hyperledger/fabric-ca-server
    container_name: ca_org1
    networks:
      - fed_fab

  ca_org2:
    image: hyperledger/fabric-ca:latest
    labels:
      service: hyperledger-fabric
    environment:
      - FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
      - FABRIC_CA_SERVER_CA_NAME=ca-org2
      - FABRIC_CA_SERVER_TLS_ENABLED=true
      - FABRIC_CA_SERVER_PORT=9050
      - FABRIC_CA_SERVER_OPERATIONS_LISTENADDRESS=0.0.0.0:19050
    ports:
      - "9050:9050"
      - "19050:19050"
    command: sh -c 'fabric-ca-server start -b admin:adminpw -d'
    volumes:
      - ../crypto-config/fabric-ca/org2:/etc/hyperledger/fabric-ca-server
    container_name: ca_org2
    networks:
      - fed_fab

  ca_orderer:
    image: hyperledger/fabric-ca:latest
    labels:
      service: hyperledger-fabric
    environment:
      - FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
      - FABRIC_CA_SERVER_CA_NAME=ca-orderer
      - FABRIC_CA_SERVER_TLS_ENABLED=true
      - FABRIC_CA_SERVER_PORT=7051
      - FABRIC_CA_SERVER_OPERATIONS_LISTENADDRESS=0.0.0.0:17051
    ports:
      - "7051:7051"
      - "17051:17051"
    command: sh -c 'fabric-ca-server start -b admin:adminpw -d'
    volumes:
      - ../crypto-config/fabric-ca/ordererOrg:/etc/hyperledger/fabric-ca-server
    container_name: ca_orderer
    networks:
      - fed_fab
  
  ca_tls:
    image: hyperledger/fabric-ca:latest
    labels:
      service: hyperledger-fabric
    environment:
      - FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
      - FABRIC_CA_SERVER_CA_NAME=ca-orderer
      - FABRIC_CA_SERVER_TLS_ENABLED=true
      - FABRIC_CA_SERVER_PORT=7054
      - FABRIC_CA_SERVER_OPERATIONS_LISTENADDRESS=0.0.0.0:17054
    ports:
      - "7054:7054"
      - "17054:17054"
    command: sh -c 'fabric-ca-server start -b admin:adminpw -d'
    volumes:
      - ../crypto-config/fabric-ca/tls-ca:/etc/hyperledger/fabric-ca-server
    container_name: ca_tls
    networks:
      - fed_fab
阅读更多
关注我

链接

分类

最新文章

归档

标签

6.8242
Android1
Android FrameWork1
Android 高级开发瓶颈突破系列课11
Android-官方文档1
Android-官方源码5
Android面试查漏补缺2
C/C++1
Flutter1
Java基础1
Kotlin-Android21
Kotlin-Android 实习25
LeetCode45
LeetCode 1013
LeetCode周赛1
Linux/UNIX系统编程手册38
OI KIWI2
OS4
PAT-(Advanced-Level)-Practice2
PAT-(Basic-Level)-Practice44
blockchain1
fedfab13
gallery1
geth1
git1
gradle1
hexo2
linux2
misc1
projects1
shell2
刷题2
动态规划1
多线程1
排序1
数据库2
牛客2
算法43
×