# pisces2-m

**Repository Path**: zhangvalue/pisces2-m

## Basic Information

- **Project Name**: pisces2-m
- **Description**: 基于Netty的LengthFieldBasedFrameDecoder自定义文件传输协议，程序由client端来触发和控制文件发送的频率，适用于传输大量的文件。


- **Primary Language**: Java
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: https://www.bbstone.cn
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 5
- **Created**: 2024-12-27
- **Last Updated**: 2024-12-27

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# pisces2-m

#### 介绍
在个人实际文件备份需求中，有大量小文件（不超过1G）需要传输，之前基于FileRegion和ChunkedFile模式实现，在Server端和Client端处理速度差积累到一定程度的时候，大量的文件传输会堵塞在server端，且没法控制传输的节凑。所以，本项目基于LengthFieldBasedFrameDecoder自定义文件传输协议，程序由client端来触发和控制文件发送的频率，适用于 _客户端接收文件处理能力比较弱的情况下_ 实现传输大量的文件。

#### 软件架构
软件架构说明
![输入图片说明](architect.png)

ChunkSlicer: server端将文件切分成一个个chunk数据块发送给client。server直接从FileInputStream一个个chunk读取数据，然后构造响应buildRsp发送给client。
RecvBuffer: client端接收到chunk数据后，先缓存起来，默认缓存大小：32 * 1024 * 1024 = 32M

#### 安装教程

1.  mvn clean install
2.  分别得到server和client的发布包（pisces2-server-bin.zip和pisces2-client-bin.zip）
3.  在需要传出文件的电脑作为server，需要接收的电脑作为client，分别解压上面两个压缩包
4. pisces2-server-bin.zip解压后的目录结构如下：
```
pisces2-server
├── bin
│   ├── server.bat
│   └── server.sh
├── conf
│   ├── config.properties
│   └── logback.xml
└── lib
    ├── commons-beanutils-1.9.3.jar
    ├── commons-codec-1.15.jar
    ├── commons-collections-3.2.2.jar
    ├── commons-configuration2-2.2.jar
    ├── commons-lang3-3.9.jar
    ├── commons-logging-1.2.jar
    ├── fast-md5-2.7.1.jar
    ├── hutool-all-5.8.1.jar
    ├── jackson-annotations-2.10.2.jar
    ├── jackson-core-2.10.2.jar
    ├── jackson-databind-2.10.2.jar
    ├── jackson-datatype-jdk8-2.10.2.jar
    ├── jackson-datatype-jsr310-2.10.2.jar
    ├── jackson-module-parameter-names-2.10.2.jar
    ├── logback-classic-1.2.3.jar
    ├── logback-core-1.2.3.jar
    ├── lombok-1.18.16.jar
    ├── netty-all-4.1.42.Final.jar
    ├── pisces2-cmm-1.0-SNAPSHOT.jar
    ├── pisces2-server-1.0-SNAPSHOT.jar
    └── slf4j-api-1.7.30.jar

3 directories, 25 files

```
5. pisces2-client-bin.zip解压后的目录结构如下：
```
pisces2-client
├── bin
│   ├── client.bat
│   └── client.sh
├── conf
│   ├── config.properties
│   └── logback.xml
└── lib
    ├── commons-beanutils-1.9.3.jar
    ├── commons-codec-1.15.jar
    ├── commons-collections-3.2.2.jar
    ├── commons-configuration2-2.2.jar
    ├── commons-lang3-3.9.jar
    ├── commons-logging-1.2.jar
    ├── fast-md5-2.7.1.jar
    ├── hutool-all-5.8.1.jar
    ├── jackson-annotations-2.10.2.jar
    ├── jackson-core-2.10.2.jar
    ├── jackson-databind-2.10.2.jar
    ├── jackson-datatype-jdk8-2.10.2.jar
    ├── jackson-datatype-jsr310-2.10.2.jar
    ├── jackson-module-parameter-names-2.10.2.jar
    ├── logback-classic-1.2.3.jar
    ├── logback-core-1.2.3.jar
    ├── lombok-1.18.16.jar
    ├── netty-all-4.1.42.Final.jar
    ├── pisces2-client-1.0-SNAPSHOT.jar
    ├── pisces2-cmm-1.0-SNAPSHOT.jar
    └── slf4j-api-1.7.30.jar

3 directories, 25 files



```
6. 配置和启动server
配置文件：pisces2-server/conf/config.properties
```
## ------------------ COMMON
## default ~/.fli_server and ~/.fli_client
#fli.server.dir=
#fli.client.dir=
## filter out pattern matched will not include in fli.idx file
fli.filter=.DS_Store,.mvn,.git
## ----------------- SERVER
## server running on which ip and port
server.port=8899
server.host=192.168.1.4
# server source dir
server.dir=/Users/bbstone/workdir/assets

```

7. 配置和启动client
配置文件：pisces2-client/conf/config.properties

```
## ------------------ COMMON
## default ~/.fli_server and ~/.fli_client
#fli.server.dir=
#fli.client.dir=

## filter out pattern matched will not include in fli.idx file, comma separated
fli.filter=.DS_Store,.mvn,.git

## ----------------- SERVER
## server running on which ip and port
server.port=8899

## change to your host
server.host=192.168.1.4

## ----------------- CLIENT
# client target dir
client.dir=H:\\pclient

## transferring file postfix
file.temp.postfix=.part

## overwrite file exits in clientDir
## false - never overwrite
## true - overwrite client file with server fileData only when client file checksum diff from server file checksum
client.file.overwrite = false


```

#### 已知问题

1.  单个client接收文件的速度还是太慢，10w不到文件，初步测试需要50分钟以上。
2.  
3.  
4.  

#### 待办
1. 设计可以同时运行多个client来提高文件接收效率
2.