:::info{title="本文主要内容"}
使用Docker本地搭建NFD Telegram机器人，替代原始的Cloudflare方法。
:::


原始[NFD](https://github.com/LloydAsp/nfd)项目是通过Cloudflare Worker进行搭建的，使用了[Cloudflare kv](https://developers.cloudflare.com/kv/)作为数据存储方案，而kv有每天1000条写入的[配额限制](https://developers.cloudflare.com/kv/platform/limits/)。此本地搭建方式使用sqlite代替kv数据库，以避免每日限额问题。

为什么不使用Livegram Bot？因为这个平台最近鬼迷心窍，开始用Livegram Bot给用户群发广告，几乎所有使用者受到影响，取消广告需要购买付费套餐，不买的话，你的机器人就变成tg的发广告工具了，着实抽象。

部署这个项目请确保你的VPS服务器可以正常访问Telegram服务器。

# 组件说明
- **NFD**：主要的机器人组件
- **Nginx Proxy Manager**：反代机器人组件的TG Webhook

# 部署流程
## 配置NFD机器人
### 基本配置
1. 创建对应文件夹，拉取项目
    ```bash
    mkdir -p /docker_data/nfd_bot && cd docker_data/nfd_bot && git clone https://github.com/LloydAsp/nfd.git src
    ```
2. 添加`pachage.json`文件
    ```bash
    cat << EOF > package.json
    {
      "name": "nfd-bot",
      "version": "1.0.0",
      "description": "Telegram message forwarding bot with anti-fraud features",
      "main": "src/index.js",
      "scripts": {
        "start": "node src/index.js",
        "dev": "nodemon src/index.js"
      },
      "dependencies": {
        "axios": "^1.4.0",
        "body-parser": "^1.20.2",
        "express": "^4.18.2",
        "sqlite3": "^5.1.6",
        "sqlite": "^4.2.1"
      },
      "devDependencies": {
        "nodemon": "^2.0.22"
      }
    }
    EOF
    ```
3. 创建`Dockerfile`
    ```bash
    cat << EOF > Dockerfile
    FROM node:16-alpine

    WORKDIR /app

    COPY package.json ./
    RUN npm install

    COPY . .

    EXPOSE 3000

    CMD ["npm", "start"]
    EOF
    ```
4. 创建`docker-compose.yaml`和`.env`文件
    ```bash
    cat << EOF > Dockerfile
    services:
      nfd-bot:
        build: .
        container_name: nfd-bot
        restart: always
        ports:
          - "${PORT:-3000}:3000"
        environment:
          - BOT_TOKEN=${BOT_TOKEN}
          - BOT_SECRET=${BOT_SECRET}
          - ADMIN_UID=${ADMIN_UID}
          - PORT=3000
          - BASE_URL=${BASE_URL}
        volumes:
          - ./src/data:/app/src/data
        network_mode: bridge
    EOF
    ```
    ```bash
    BOT_TOKEN='替换为你自己的Bot Token （Bot Father获取）'
    BOT_SECRET='UUID生成或替换为你自己的强密码'
    ADMIN_UID=访问https://t.me/myidbot获取你自己账户的id
    BASE_URL=https://example.com/tg # 注意最后不要加'/'
    PORT=65035 # 替换你自己的端口
    ```
    :::note{title="UUID生成"}
    <iframe width='100%' height='300px' src='https://blog.useforall.com/c/uuid_generator'>
    :::
5. 创建`src/index.js`，`nano src/index.js`
    ```javascript
    const express = require('express');
    const bodyParser = require('body-parser');
    const axios = require('axios');
    const fs = require('fs');
    const path = require('path');
    const sqlite3 = require('sqlite3');
    const { open } = require('sqlite');

    const app = express();
    app.use(bodyParser.json());

    // 从环境变量获取配置
    const TOKEN = process.env.BOT_TOKEN;
    const SECRET = process.env.BOT_SECRET;
    const ADMIN_UID = process.env.ADMIN_UID;
    const PORT = process.env.PORT || 3000;
    const WEBHOOK_PATH = process.env.WEBHOOK_PATH || '/endpoint';
    const NOTIFY_INTERVAL = 3600 * 1000;
    const enable_notification = true;

    // 数据文件路径
    const FRAUD_DB_PATH = path.join(__dirname, 'data/fraud.db');
    const NOTIFICATION_PATH = path.join(__dirname, 'data/notification.txt');
    const START_MSG_PATH = path.join(__dirname, 'data/startMessage.md');
    const DB_PATH = path.join(__dirname, 'data/nfd.sqlite');

    // SQLite 数据库连接
    let db;

    // 初始化数据库
    async function initDatabase() {
      db = await open({
        filename: DB_PATH,
        driver: sqlite3.Database
      });

      // 创建表
      await db.exec(`
        CREATE TABLE IF NOT EXISTS kv_store (
          key TEXT PRIMARY KEY,
          value TEXT,
          timestamp INTEGER
        )
      `);

      console.log('Database initialized');
    }

    // 封装 KV 操作的函数
    async function kvGet(key) {
      const row = await db.get('SELECT value FROM kv_store WHERE key = ?', [key]);
      return row ? JSON.parse(row.value) : null;
    }

    async function kvPut(key, value) {
      const jsonValue = JSON.stringify(value);
      await db.run(
        'INSERT OR REPLACE INTO kv_store (key, value, timestamp) VALUES (?, ?, ?)',
        [key, jsonValue, Date.now()]
      );
    }

    /**
     * 返回 Telegram API URL
     */
    function apiUrl(methodName, params = null) {
      let query = '';
      if (params) {
        const searchParams = new URLSearchParams();
        Object.entries(params).forEach(([key, value]) => {
          searchParams.append(key, value);
        });
        query = '?' + searchParams.toString();
      }
      return `https://api.telegram.org/bot${TOKEN}/${methodName}${query}`;
    }

    /**
     * 向 Telegram API 发送请求
     */
    async function requestTelegram(methodName, body, params = null) {
      try {
        const response = await axios.post(apiUrl(methodName, params), body);
        return response.data;
      } catch (error) {
        console.error(`Error calling Telegram API ${methodName}:`, error.message);
        return { ok: false, error: error.message };
      }
    }

    function makeReqBody(body) {
      return body;
    }

    function sendMessage(msg = {}) {
      return requestTelegram('sendMessage', makeReqBody(msg));
    }

    function copyMessage(msg = {}) {
      return requestTelegram('copyMessage', makeReqBody(msg));
    }

    function forwardMessage(msg) {
      return requestTelegram('forwardMessage', makeReqBody(msg));
    }

    /**
     * 处理 webhook 请求
     */
    app.post(WEBHOOK_PATH, async (req, res) => {
      // 验证 secret token
      if (req.headers['x-telegram-bot-api-secret-token'] !== SECRET) {
        return res.status(403).send('Unauthorized');
      }

      const update = req.body;

      // 异步处理 update
      onUpdate(update).catch(err => {
        console.error('Error processing update:', err);
      });

      return res.status(200).send('Ok');
    });

    /**
     * 处理传入的 Update
     */
    async function onUpdate(update) {
      if ('message' in update) {
        await onMessage(update.message);
      }
    }

    /**
     * 处理传入的 Message
     */
    async function onMessage(message) {
      if (message.text === '/start') {
        let startMsg = await readFile(START_MSG_PATH);
        return sendMessage({
          chat_id: message.chat.id,
          text: startMsg,
        });
      }

      if (message.chat.id.toString() === ADMIN_UID) {
        if (!message?.reply_to_message?.chat) {
          return sendMessage({
            chat_id: ADMIN_UID,
            text: '使用方法，回复转发的消息，并发送回复消息，或者`/block`、`/unblock`、`/checkblock`等指令'
          });
        }

        if (/^\/block$/.exec(message.text)) {
          return handleBlock(message);
        }

        if (/^\/unblock$/.exec(message.text)) {
          return handleUnBlock(message);
        }

        if (/^\/checkblock$/.exec(message.text)) {
          return checkBlock(message);
        }

        let guestChantId = await kvGet(`msg-map-${message?.reply_to_message.message_id}`);
        return copyMessage({
          chat_id: guestChantId,
          from_chat_id: message.chat.id,
          message_id: message.message_id,
        });
      }

      return handleGuestMessage(message);
    }

    async function handleGuestMessage(message) {
      let chatId = message.chat.id;
      let isblocked = await kvGet(`isblocked-${chatId}`);

      if (isblocked) {
        return sendMessage({
          chat_id: chatId,
          text: 'Your are blocked'
        });
      }

      let forwardReq = await forwardMessage({
        chat_id: ADMIN_UID,
        from_chat_id: message.chat.id,
        message_id: message.message_id
      });

      console.log(JSON.stringify(forwardReq));

      if (forwardReq.ok) {
        await kvPut(`msg-map-${forwardReq.result.message_id}`, chatId);
      }

      return handleNotify(message);
    }

    async function handleNotify(message) {
      // 先判断是否是诈骗人员，如果是，则直接提醒
      // 如果不是，则根据时间间隔提醒：用户id，交易注意点等
      let chatId = message.chat.id;

      if (await isFraud(chatId)) {
        return sendMessage({
          chat_id: ADMIN_UID,
          text: `检测到骗子，UID${chatId}`
        });
      }

      if (enable_notification) {
        let lastMsgTime = await kvGet(`lastmsg-${chatId}`);

        if (!lastMsgTime || Date.now() - lastMsgTime > NOTIFY_INTERVAL) {
          await kvPut(`lastmsg-${chatId}`, Date.now());

          return sendMessage({
            chat_id: ADMIN_UID,
            text: await readFile(NOTIFICATION_PATH)
          });
        }
      }
    }

    async function handleBlock(message) {
      let guestChantId = await kvGet(`msg-map-${message.reply_to_message.message_id}`);

      if (guestChantId === ADMIN_UID) {
        return sendMessage({
          chat_id: ADMIN_UID,
          text: '不能屏蔽自己'
        });
      }

      await kvPut(`isblocked-${guestChantId}`, true);

      return sendMessage({
        chat_id: ADMIN_UID,
        text: `UID:${guestChantId}屏蔽成功`,
      });
    }

    async function handleUnBlock(message) {
      let guestChantId = await kvGet(`msg-map-${message.reply_to_message.message_id}`);
      await kvPut(`isblocked-${guestChantId}`, false);

      return sendMessage({
        chat_id: ADMIN_UID,
        text: `UID:${guestChantId}解除屏蔽成功`,
      });
    }

    async function checkBlock(message) {
      let guestChantId = await kvGet(`msg-map-${message.reply_to_message.message_id}`);
      let blocked = await kvGet(`isblocked-${guestChantId}`);

      return sendMessage({
        chat_id: ADMIN_UID,
        text: `UID:${guestChantId}` + (blocked ? '被屏蔽' : '没有被屏蔽')
      });
    }

    /**
     * 发送纯文本消息
     */
    async function sendPlainText(chatId, text) {
      return sendMessage({
        chat_id: chatId,
        text
      });
    }

    /**
     * 设置 webhook
     */
    app.get('/registerWebhook', async (req, res) => {
      const BASE_URL = process.env.BASE_URL || `http://localhost:${PORT}`;
      const webhookUrl = `${BASE_URL}${WEBHOOK_PATH}`;

      try {
        const r = await axios.get(apiUrl('setWebhook', { 
          url: webhookUrl, 
          secret_token: SECRET 
        }));

        if (r.data.ok) {
          return res.send('Webhook registered successfully');
        } else {
          return res.status(400).send(JSON.stringify(r.data, null, 2));
        }
      } catch (error) {
        return res.status(500).send(`Error: ${error.message}`);
      }
    });

    /**
     * 删除 webhook
     */
    app.get('/unRegisterWebhook', async (req, res) => {
      try {
        const r = await axios.get(apiUrl('setWebhook', { url: '' }));

        if (r.data.ok) {
          return res.send('Webhook unregistered successfully');
        } else {
          return res.status(400).send(JSON.stringify(r.data, null, 2));
        }
      } catch (error) {
        return res.status(500).send(`Error: ${error.message}`);
      }
    });

    /**
     * 检查是否是骗子
     */
    async function isFraud(id) {
      id = id.toString();
      const fraudDb = await readFile(FRAUD_DB_PATH);
      let arr = fraudDb.split('\n').filter(v => v);
      console.log(JSON.stringify(arr));
      let flag = arr.filter(v => v === id).length !== 0;
      console.log(flag);
      return flag;
    }

    /**
     * 读取文件辅助函数
     */
    function readFile(filePath) {
      return new Promise((resolve, reject) => {
        fs.readFile(filePath, 'utf8', (err, data) => {
          if (err) {
            console.error(`Error reading file ${filePath}:`, err);
            reject(err);
          } else {
            resolve(data);
          }
        });
      });
    }

    // 启动服务器
    async function startServer() {
      // 初始化数据库
      await initDatabase();

      // 启动 Express 服务器
      app.listen(PORT, () => {
        console.log(`NFD Bot server running on port ${PORT}`);
        console.log(`Webhook path: ${WEBHOOK_PATH}`);
      });
    }

    // 启动应用
    startServer().catch(err => {
      console.error('Failed to start server:', err);
      process.exit(1);
    });
    ```
6. 启动服务
    ```bash
    docker compose up -d
    ```
&nbsp;
## 配置Nginx Proxy Manager
1. 随便找一个已经反代了的服务(这里假设是`https://example.com`服务)，前往`Custom locations`，如图添加一个`location`
    ![image.png](https://lsky.hk.tencent.useforall.com/Blog/2025/03/15/67d52ece06a1a.png)
2. 访问`'https://example.com/tg/registerWebhook`。
## 其他
1. 验证是否成功
    1. 如果Bot发送消息给你了，那就代表成功了。
    2. 或者运行`curl "https://api.telegram.org/bot替换为你的BotToken/getWebhookInfo"`，返回如下结果就算成功。
        ```
        {"ok":true,"result":{"url":"https://example/tg/endpoint","has_custom_certificate":false,"pending_update_count":0,"max_connections":40,"ip_address":"xx.xx.xx.xx"}}
        ```

Docker搭建Node Foward Bot(NFD) TG机器人

:::info{title="本文主要内容"}
使用NPM来创建一个自己的Huggingface镜像站，并设置URL Prefix来防止被自动采集后被被人使用。
:::


众所周知，Huggingface不能国内直接访问，同时镜像站又存在速度限制，例如`hf-mirror`最高我这只有`6MB/s`。`hf-mirror`虽然开源了镜像站代码，但是实测无法运行（主要由于本人不会使用Caddy）。此外使用VPN虽然可以解决无法访问的情况，但是一直存在一个问题就是模型会越下越慢，哪怕开启TUN模式也没用，目前也没找到解决办法，所以至此萌生了使用NPM反代一个自己的镜像站。目前实测下载模型不会越下越慢，同时所有网页功能均可使用，包括登录注册等。
# 组件说明
- **Nginx Proxy Manager**: 提供反向代理功能。搭建参考：[搭建 Nginx Proxy Manager](https://blog.useforall.com/post/3#%E9%85%8D%E7%BD%AE%20Nginx%20Proxy%20Manager)

# 部署流程
## 反代Huggingface主站和LFS CDN下载节点
1. 如图配置，这里假设你用的域名是`example.com`：
    ![image.png](https://lsky.hk.tencent.useforall.com/Blog/2025/03/01/67c2e3a81cd93.png)
2. 前往Custom Location设置直接访问`https://example.com/`时判断是否有我们自己设置的`cookie`，没有的话，不允许访问：
    ```
    # 检查 Cookie 是否存在
    if ($http_cookie !~* "access_granted=true") { # key-value pair请一定跟后续配置的一样
        return 403; # 或者 401, 502，根据你的需求选择合适的错误码
    }

    # 如果 Cookie 存在，并且 URI 不是以 /jZPU8pzjCweCDUiXxGB9B/ 开头，则内部重写。
    # 请注意替换为`设置实际访问的反代链接`部分设置的location，这里只是以我自己的location举例
    if ($request_uri !~ "^/jZPU8pzjCweCDUiXxGB9B/") {
        rewrite ^/(.*)$ /jZPU8pzjCweCDUiXxGB9B/$1 last;
        # 使用 'last' 指令进行内部重写，并重新查找 location
    }
    ```
    ![image.png](https://lsky.hk.tencent.useforall.com/Blog/2025/03/01/67c2e48492279.png)
3. 反代下载节点，如图配置（请自行修改你需要的`location`）：
    ```
    proxy_ssl_server_name on;
    proxy_ssl_name cdn-lfs-us-1.hf.co;
    proxy_set_header Host cdn-lfs-us-1.hf.co;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_buffering off;
    proxy_redirect off;
    ```
    ![image.png](https://lsky.hk.tencent.useforall.com/Blog/2025/03/01/67c2e36be707a.png)
    :::note{title="注"}
    1. `location`可以自行设置，这里代表的含义是当你访问`https://example.com/b3k3EnxpRbqvuomRASVE3/repos/***.bin`下载LFS文件的时候，实际上访问的是`https://cdn-lfs-us-1.hf.co/repos/***.bin`，但是这不代表你是通过原Huggingface CDN连接下载，而是你是通过你的VPS->VPS访问文件->VPS传输回来给你的。
    :::
## 设置实际访问的反代链接
1. 还是在Custom Location，如图配置：
    ```
    proxy_ssl_server_name on;
    proxy_ssl_name huggingface.co;
    proxy_set_header Host huggingface.co;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_buffering off;
    proxy_redirect https://cdn-lfs-us-1.hf.co https://example.com/b3k3EnxpRbqvuomRASVE3; 
    proxy_redirect https://huggingface.co https://example.com/jZPU8pzjCweCDUiXxGB9B/;
    add_header Set-Cookie "access_granted=true; Path=/;"; # Path=/ 表示 Cookie 对整个域名有效。可以修改为你自己想要的key-value pair。
    ```
    ![image.png](https://lsky.hk.tencent.useforall.com/Blog/2025/03/01/67c2e65febc0a.png)
    
    :::note{title="注"}
    1. `proxy_redirect https://cdn-lfs-us-1.hf.co https://example.com/b3k3EnxpRbqvuomRASVE3;`：修改`exmaple.com`为你自己设置域名，修改`/b3k3EnxpRbqvuomRASVE3`为`上一部分第三步`设置的location，记得不要最后的`/`。
    2. `proxy_redirect https://huggingface.co https://example.com/jZPU8pzjCweCDUiXxGB9B/;`：修改`exmaple.com`为你自己设置域名，修改`/jZPU8pzjCweCDUiXxGB9B/`为`此步骤`设置的location。
    :::
2. 结束。至此，当正确访问`https://example.com/jZPU8pzjCweCDUiXxGB9B/`后，浏览器会设置一个`cookie`，也就是`access_granted=true`，此后哪怕没有输入正确的location，也可以正常访问，因为判断到浏览器有对应的`cookie`。注意这个cookie是会话级的，关闭窗口一段时间后会失效，要重新访问正确的连接才可以。

使用Nginx Proxy Manager搭建自己的Huggingface镜像站

:::info{title="本文主要内容"}
由于`OPNsense 24.1`默认配置备份只支持本地和Goolge Drive两种备份方式，本文主要是给`OPNSense`添加`WebDAV`备份功能。
:::


# 配置流程
## 添加WebDAV备份插件
1. 进入`OPNSense`，打开`安全shell`设置，ssh连接进入系统shell。
    ![CleanShot 2025-02-26 at 12.53.16@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/26/67be9e56928a1.png)
2. 创建文件夹 `mkdir -p /usr/local/opnsense/mvc/app/models/OPNsense/Backup/` 后，添加核心实现逻辑代码。
    ```bash
    cat << 'EOF' > /usr/local/opnsense/mvc/app/library/OPNsense/Backup/WebDAV.php
    <?php
    ///usr/local/opnsense/mvc/app/library/OPNsense/Backup/WebDAV.php
    /*
     * Copyright (C) 2018 Deciso B.V.
     * Copyright (C) 2018 Fabian Franz
     * All rights reserved.
     *
     * Redistribution and use in source and binary forms, with or without
     * modification, are permitted provided that the following conditions are met:
     *
     * 1. Redistributions of source code must retain the above copyright notice,
     *    this list of conditions and the following disclaimer.
     *
     * 2. Redistributions in binary form must reproduce the above copyright
     *    notice, this list of conditions and the following disclaimer in the
     *    documentation and/or other materials provided with the distribution.
     *
     * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
     * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
     * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
     * AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
     * OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
     * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
     * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
     * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
     * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
     * POSSIBILITY OF SUCH DAMAGE.
     */

    namespace OPNsense\Backup;

    use OPNsense\Core\Config;

    /**
     * Class WebDAV backup
     * @package OPNsense\Backup
     */
    class WebDAV extends Base implements IBackupProvider
    {
        /**
         * get required (user interface) fields for backup connector
         * @return array configuration fields, types and description
         */
        public function getConfigurationFields()
        {
            $fields = array(
                array(
                    "name" => "enabled",
                    "type" => "checkbox",
                    "label" => gettext("Enable"),
                    "value" => null
                ),
                array(
                    "name" => "url",
                    "type" => "text",
                    "label" => gettext("URL"),
                    "help" => gettext("The Base URL to WebDAV without trailing slash. For example: https://dav.example.com"),
                    "value" => null
                ),
                array(
                    "name" => "user",
                    "type" => "text",
                    "label" => gettext("User Name"),
                    "help" => gettext("The name you use for logging into your WebDAV account"),
                    "value" => null
                ),
                array(
                    "name" => "password",
                    "type" => "password",
                    "label" => gettext("Password"),
                    "help" => gettext("The password you use for logging into your WebDAV account"),
                    "value" => null
                ),
                array(
                    "name" => "password_encryption",
                    "type" => "password",
                    "label" => gettext("Encryption Password (Optional)"),
                    "help" => gettext("A password to encrypt your configuration"),
                    "value" => null
                ),
                array(
                    "name" => "backupdir",
                    "type" => "text",
                    "label" => gettext("Directory Name without leading slash, starting from user's root"),
                    "value" => 'OPNsense-Backup'
                )
            );
            $webdav = new WebDAVSettings();
            foreach ($fields as &$field) {
                $field['value'] = (string)$webdav->getNodeByReference($field['name']);
            }
            return $fields;
        }

        /**
         * backup provider name
         * @return string user friendly name
         */
        public function getName()
        {
            return gettext("WebDAV");
        }

        /**
         * validate and set configuration
         * @param array $conf configuration array
         * @return array of validation errors when not saved
         * @throws \OPNsense\Base\ModelException
         * @throws \ReflectionException
         */
        public function setConfiguration($conf)
        {
            $webdav = new WebDAVSettings();
            $this->setModelProperties($webdav, $conf);
            $validation_messages = $this->validateModel($webdav);
            if (empty($validation_messages)) {
                $webdav->serializeToConfig();
                Config::getInstance()->save();
            }
            return $validation_messages;
        }

        /**
         * perform backup
         * @return array filelist
         * @throws \OPNsense\Base\ModelException
         * @throws \ReflectionException
         */
        public function backup()
        {
            $cnf = Config::getInstance();
            $webdav = new WebDAVSettings();
            if ($cnf->isValid() && !empty((string)$webdav->enabled)) {
                $config = $cnf->object();
                $url = (string)$webdav->url;
                $username = (string)$webdav->user;
                $password = (string)$webdav->password;
                $backupdir = (string)$webdav->backupdir;
                $crypto_password = (string)$webdav->password_encryption;
                $hostname = $config->system->hostname . '.' . $config->system->domain;
                $configname = 'config-' . $hostname . '-' .  date('Y-m-d_H_i_s') . '.xml';
                // backup source data to local strings (plain/encrypted)
                $confdata = file_get_contents('/conf/config.xml');
                if (!empty($crypto_password)) {
                    $confdata = $this->encrypt($confdata, $crypto_password);
                }
                // Check if destination directory exists, create (full path) if not
                try {
                    $this->create_directory($url, $username, $password, $backupdir);
                } catch (\Exception $e) {
                    return array();
                }
                try {
                    $this->upload_file_content(
                        $url,
                        $username,
                        $password,
                        $backupdir,
                        $configname,
                        $confdata
                    );
                    // do not list directories
                    return array_filter(
                        $this->listFiles($url, $username, $password, "/$backupdir/", false),
                        function ($filename) {
                            return (substr($filename, -1) !== '/');
                        }
                    );
                } catch (\Exception $e) {
                    return array();
                }
            }
        }

        /**
         * dir listing
         * @param string $url remote location
         * @param string $username username
         * @param string $password password to use
         * @param string $directory location to list
         * @param bool $only_dirs only list directories
         * @return array
         * @throws \Exception
         */
        public function listFiles($url, $username, $password, $directory = '/', $only_dirs = true)
        {
            $result = $this->curl_request(
                "$url$directory",
                $username,
                $password,
                'PROPFIND',
                "Error while fetching filelist from WebDAV '{$directory}' path"
            );
            //remove line breaks from xml string
            $xml = preg_replace("/\r?\n/", '', $result['response']);

            // workaround - simplexml seems to be broken when using namespaces - remove them.
            //$xml = str_replace(['<D:', '</D:', '<d:', '</d:', '<lp1:', '</lp1:'], ['<', '</', '<', '</', '<', '</'], $xml);
            // better workaround: remove any namespace
            $xml = preg_replace('/\s+xmlns:[^=]+="[^"]*"/', '', $xml);
            $xml = preg_replace('/\b(\w+)\:/i', '', $xml);

            $xml = simplexml_load_string($xml);
            $ret = array();

            //parse URL for a check if path exists
            $parsedUrl = parse_url($url);

            foreach ($xml->children() as $response) {
                // d:response
                if ($response->getName() == 'response') {

                    $fileurl = (string)$response->href;

                    //check if URL has a path
                    if (isset($parsedUrl['path'])) {
                        //URL DOES have a path - extracting path from fileurl
                        $dirname = explode($parsedUrl['path'], $fileurl, 2)[1];
                    }
                    else {
                        //URL does NOT have a path
                        $dirname = $fileurl;
                    }

                    if (
                        $response->propstat->prop->resourcetype->children()->count() > 0 &&
                        $response->propstat->prop->resourcetype->children()[0]->getName() == 'collection' &&
                        $only_dirs
                    ) {

                        $ret[] = $dirname;

                    } elseif (!$only_dirs) {

                        $ret[] = $dirname;

                    }
                }
            }
            return $ret;
        }

        /**
         * upload file
         * @param string $url remote location
         * @param string $username remote user
         * @param string $password password to use
         * @param string $backupdir remote directory
         * @param string $filename filename to use
         * @param string $local_file_content contents to save
         * @throws \Exception when upload fails
         */
        public function upload_file_content($url, $username, $password, $backupdir, $filename, $local_file_content)
        {
            $this->curl_request(
                $url . "/$backupdir/$filename",
                $username,
                $password,
                'PUT',
                'cannot execute PUT',
                $local_file_content
            );
        }

        /**
         * create new remote directory if doesn't exist
         * @param string $url remote location
         * @param string $username remote user
         * @param string $password password to use
         * @param string $backupdir remote directory
         * @throws \Exception when create dir fails
         */
        public function create_directory($url, $username, $password, $backupdir)
        {
            $parent_path = dirname($backupdir);

            try {
                $directories = $this->listFiles($url, $username, $password, "/{$parent_path}");
            } catch (\Exception $e) {
                if ($backupdir == ".") {
                    // We cannot create root, if we reached here there's some other problem
                    syslog(LOG_ERR, "Check WebDAV configuration parameters");
                    return false;
                }
                // If error assume dir doesn't exist. Create parent folder
                if ($this->create_directory($url, $username, $password, $parent_path) === false) {
                    throw new \Exception();
                }
            }

            // if path exists ok
            if (in_array("/{$backupdir}/", $directories)) {
                return;
            }

            // create backupdir, because path does not exist
            $this->curl_request(
                $url . "/{$backupdir}",
                $username,
                $password,
                'MKCOL',
                'cannot execute MKCOL'
            );
        }

        /**
         * @param string $url remote location
         * @param string $username remote user
         * @param string $password password to use
         * @param string $method http method, PUT, GET, ...
         * @param string $error_message message to log on failure
         * @param null|string $postdata http body
         * @param array $headers HTTP headers
         * @return array response status
         * @throws \Exception when request fails
         */
        public function curl_request(
            $url,
            $username,
            $password,
            $method,
            $error_message,
            $postdata = null,
            $headers = array('User-Agent: OPNsense Firewall')
        ) {
            //workaround for Hetzner Storagebox
            if ($method == "PROPFIND") {
            array_push($headers, 'Depth: 1');
            }
        $curl = curl_init();
            curl_setopt_array($curl, array(
                CURLOPT_URL => $url,
                CURLOPT_CUSTOMREQUEST => $method, // Create a file in WebDAV is PUT
                CURLOPT_RETURNTRANSFER => true, // Do not output the data to STDOUT
                CURLOPT_VERBOSE => 0,           // same here
                CURLOPT_MAXREDIRS => 0,         // no redirects
                CURLOPT_TIMEOUT => 60,          // maximum time: 1 min
                CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
                CURLOPT_USERPWD => $username . ":" . $password,
                CURLOPT_HTTPHEADER => $headers
            ));
            if ($postdata != null) {
                curl_setopt($curl, CURLOPT_POSTFIELDS, $postdata);
            }
            $response = curl_exec($curl);
            $err = curl_error($curl);
            $info = curl_getinfo($curl);
            if (!($info['http_code'] == 200 || $info['http_code'] == 207 || $info['http_code'] == 201) || $err) {
                syslog(LOG_ERR, $error_message);
                syslog(LOG_ERR, json_encode($info));
                throw new \Exception();
            }
            curl_close($curl);
            return array('response' => $response, 'info' => $info);
        }

        /**
         * Is this provider enabled
         * @return boolean enabled status
         * @throws \OPNsense\Base\ModelException
         * @throws \ReflectionException
         */
        public function isEnabled()
        {
            $webdav = new WebDAVSettings();
            return (string)$webdav->enabled === "1";
        }
    }
    'EOF'
    ```
    ```bash
    cat << 'EOF' > /usr/local/opnsense/mvc/app/models/OPNsense/Backup/WebDAVSettings.php
    <?php
    ///usr/local/opnsense/mvc/app/models/OPNsense/Backup/WebDAVSettings.php
    /**
     *    Copyright (C) 2018 Fabian Franz
     *
     *    All rights reserved.
     *
     *    Redistribution and use in source and binary forms, with or without
     *    modification, are permitted provided that the following conditions are met:
     *
     *    1. Redistributions of source code must retain the above copyright notice,
     *       this list of conditions and the following disclaimer.
     *
     *    2. Redistributions in binary form must reproduce the above copyright
     *       notice, this list of conditions and the following disclaimer in the
     *       documentation and/or other materials provided with the distribution.
     *
     *    THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
     *    INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
     *    AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
     *    AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
     *    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
     *    SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
     *    INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
     *    CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
     *    ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
     *    POSSIBILITY OF SUCH DAMAGE.
     *
     */

    namespace OPNsense\Backup;

    use OPNsense\Base\BaseModel;

    /**
     * Class WebDAV
     * @package Backup
     */
    class WebDAVSettings extends BaseModel
    {
    }
    'EOF'
    ```
3. 添加`WebDAVSettings.xml`，这一步让在`OPNSense`备份中显示`WebDAV`配置界面。
    ```bash
   cat << 'EOF' > /usr/local/opnsense/mvc/app/models/OPNsense/Backup/WebDAVSettings.xml
    <model>
        <mount>//system/backup/webdav</mount>
        <version>1.0.0</version>
        <description>OPNsense WebDAV Backup Settings</description>
        <items>
            <enabled type="BooleanField">
                <default>0</default>
                <Required>Y</Required>
            </enabled>
            <url type="TextField">
                <Required>N</Required>
                <mask>/^https?:\/\/.*[^\/]$/</mask>
                <ValidationMessage>The URL must be valid without a trailing slash. For example: https://dav.example.com</ValidationMessage>
                <Constraints>
                    <check001>
                        <ValidationMessage>A URL for the WebDAV server must be set.</ValidationMessage>
                        <type>DependConstraint</type>
                        <addFields>
                            <field1>enabled</field1>
                        </addFields>
                    </check001>
                </Constraints>
            </url>
            <user type="TextField">
                <Constraints>
                    <check001>
                        <ValidationMessage>A user for the WebDAV server must be set.</ValidationMessage>
                        <type>DependConstraint</type>
                        <addFields>
                            <field1>enabled</field1>
                        </addFields>
                    </check001>
                </Constraints>
            </user>
            <password type="TextField">
                <Constraints>
                    <check001>
                        <ValidationMessage>A password for the WebDAV server must be set.</ValidationMessage>
                        <type>DependConstraint</type>
                        <addFields>
                            <field1>enabled</field1>
                        </addFields>
                    </check001>
                </Constraints>
            </password>
            <password_encryption type="TextField">
                <Required>N</Required>
            </password_encryption>
            <backupdir type="TextField">
                <Required>Y</Required>
                <mask>/^([\w%+\-]+\/)*[\w+%\-]+$/</mask>
                <default>OPNsense-Backup</default>
                <ValidationMessage>The Backup Directory can only consist of alphanumeric characters, dash, underscores and slash. No leading or trailing slash.</ValidationMessage>
            </backupdir>
        </items>
    </model>
    'EOF'
    ```
4. 进入`OPNSense`，`系统 -> 配置 -> 备份 -> 滑到最后`。按需配置即可。
    ![CleanShot 2025-02-26 at 12.59.24@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/26/67be9ffc9e125.png)

OPNSense添加WebDAV备份插件

:::info{title="本文主要内容"}
本文主要介绍了如何使用Docker搭建GitLab代码服务器，并且解决无法使用标准端口（443）可能存在的问题，最后用Nginx Proxy Manager进行反向代理。
:::


# 组件说明
- **GitLab**：代码托管平台，可以自部署。
- **Nginx Proxy Manager**: 提供反向代理功能

# 部署流程
## 配置GitLab
1. `mkdir docker_data && cd docker_data && mkdir gitlab && cd gitlab && nano docker-compose.yaml`
    添加以下内容：
    ```yaml
    services:
      gitlab:
        image: gitlab/gitlab-ee:17.4.2-ee.0
        container_name: gitlab
        restart: always
        ports:
          - 65007:80
          - 65008:443
          - 65009:22
        volumes:
          - ./config:/etc/gitlab
          - ./logs:/var/log/gitlab
          - ./data:/var/opt/gitlab
        shm_size: '256m'
        network_mode: bridge
    ```
2. 启动服务：
    ```bash
    sudo docker compose up -d
    ```
3. 修改配置。`nano config/gitlab.rb`
    ```bash
    # 替换eternal_url为域名（非标端口，443端口未开放的情况）
    sudo sed -i "s|# external_url 'GENERATED_EXTERNAL_URL'|external_url 'https://example.com:4433'|" config/gitlab.rb
    sudo sed -i "s|# letsencrypt\['enable'\] = nil|letsencrypt['enable'] = false|" config/gitlab.rb
    sudo sed -i "s|# nginx\['listen_port'\] = nil|nginx\['listen_port'\] = 80|" config/gitlab.rb
    sudo sed -i "s|# nginx\['listen_https'\] = nil|nginx\['listen_https'\] = false|" config/gitlab.rb
    sudo docker compose up -d --force-recreate && sudo docker compose exec -it gitlab grep 'Password:' /etc/gitlab/initial_root_password
    ```
    :::note{title="注"}
    1. `external_url` 为`http`时，默认监听80。如果需要使用非标端口，例192.168.1.67:8008，那么`docker-compose.yaml`里端口映射改为65007:8008
    2. 如果为`https`，经过测试，默认只能是443，例如上面指定4433端口但实际还是监听443。同时当为https时，没有提供有效的证书它甚至会去申请Let’s Encrypt的证书，这一定是会失败的（因为根本没配置`acme`）。此时就需要修改`nginx['listen_port']`为`80`，同时关闭https监听（因为我们要自己使用nginx反代）（相当于只保留了http服务）
    :::
4. 开心版（网络收集，不确认是否有效）
    1. 先创建一个`license.rb` 。`mkdir crack && cd crack && nano license.rb`
        ```ruby
        require "openssl"
        require "gitlab/license"

        key_pair = OpenSSL::PKey::RSA.generate(2048)
        File.open("license_key", "w") { |f| f.write(key_pair.to_pem) }

        public_key = key_pair.public_key
        File.open("license_key.pub", "w") { |f| f.write(public_key.to_pem) }

        private_key = OpenSSL::PKey::RSA.new File.read("license_key")
        Gitlab::License.encryption_key = private_key

        license = Gitlab::License.new
        license.licensee = {
          "Name" => "修改为你想叫的名字",
          "Company" => "修改为你想叫的名字",
          "Email" => "修改为你想要的邮箱@example.com",
        }
        license.starts_at = Date.new(2024, 1, 1) # 开始时间
        license.expires_at = Date.new(2050, 12, 31) # 结束时间
        license.notify_admins_at = Date.new(2049, 12, 31)
        license.notify_users_at = Date.new(2049, 12, 31)
        license.block_changes_at = Date.new(2050, 12, 1) 
        license.restrictions = {
          active_user_count: 100000,
          plan: "ultimate",
          id: 1,
          subscription_id: 1,
        }

        puts "License:"
        puts license

        data = license.export
        puts "Exported license:"
        puts data
        File.open("GitLabBV.gitlab-license", "w") { |f| f.write(data) }

        public_key = OpenSSL::PKey::RSA.new File.read("license_key.pub")
        Gitlab::License.encryption_key = public_key

        data = File.read("GitLabBV.gitlab-license")
        $license = Gitlab::License.import(data)

        puts "Imported license:"
        puts $license

        unless $license
          raise "The license is invalid."
        end

        if $license.restricted?(:active_user_count)
          active_user_count = 10000
          if active_user_count > $license.restrictions[:active_user_count]
            raise "The active user count exceeds the allowed amount!"
          end
        end

        if $license.notify_admins?
          puts "The license is due to expire on #{$license.expires_at}."
        end

        if $license.notify_users?
          puts "The license is due to expire on #{$license.expires_at}."
        end

        module Gitlab
          class GitAccess
            def check(cmd, changes = nil)
              if $license.block_changes?
                return build_status_object(false, "License expired")
              end
            end
          end
        end

        puts "This instance of GitLab Enterprise Edition is licensed to:"
        $license.licensee.each do |key, value|
          puts "#{key}: #{value}"
        end

        if $license.expired?
          puts "The license expired on #{$license.expires_at}"
        elsif $license.will_expire?
          puts "The license will expire on #{$license.expires_at}"
        else
          puts "The license will never expire."
        end
        ```
    2. 运行ruby镜像生成证书。`sudo docker run -it --rm -v ./crack:/crack ruby:latest bash`。会进入到**容器内的bash界面**，输入以下指令：
        ```
        cd /crack && gem install gitlab-license && ruby license.rb
        ```
        1. 会额外生成三个文件`GitLabBV.gitlab-license`, `license_key`, `license_key.pub`。
        2. 退出容器，进入crack文件夹，把`license_key.pub`拷贝到容器内。
            ```bash
            sudo docker compose cp license_key.pub gitlab:/opt/gitlab/embedded/service/gitlab-rails/.license_encryption_key.pub
            ```
    3. 按图片操作。`cat crack/GitLabBV.gitlab-license`。
        ![CleanShot 2024-10-25 at 13.46.47@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd817440bed.png)
## 配置Nginx Proxy Manager
1. 没什么难度，就反代80端口就行。不再赘述了这里。

## 其他
1. 非标端口进行ssh推送
    ```bash
    git clone ssh://git@exmaple.com:65009/develop/test.git
    ```

Docker搭建GitLab并使用Nginx Proxy Manager反代

:::info{title="本文主要内容"}
在PVE里，使用两种方法实现透明代理。通过创建CT虚拟机或者创建OpenWRT虚拟机（笔者目前使用的方法）。涉及到的文件都是免费的，请联系笔者获取。
:::


> 2025-3-1：添加OPNSense UDP说明。
# 部署流程
## CT虚拟机方式
1. 下载debian/ubuntu CT template。
    ![CleanShot 2024-09-23 at 21.55.09@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd6daca2ba9.png)
2. 创建完成后，开始直通设备。请确认自己PVE安装的版本，PVE8.2版本开始，tun设备直通和之前版本不一样了。
    :::note{title="PVE < 8.2"}
    确认新创建的CT机器的ID，打开PVE自身的Shell，输入`nano /etc/pve/lxc/容器id.conf`
    ```conf
    # 添加最后
    lxc.apparmor.profile: unconfined
    lxc.cgroup.devices.allow: a
    lxc.cap.drop:
    lxc.cgroup2.devices.allow: c 10:200 rwm
    lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file
    ```
    :::
    :::note{title="PVE ≥ 8.2"}
    添加一个Device Passthrough
    ![CleanShot 2024-09-30 at 17.38.22@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd6fd2089d0.png)
    :::
3. 完成后，启动CT虚拟机，从PVE进入虚拟机的Console，手动开启root登录权限 
    ```bash
    sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config
    ```
4. [下载Clash文件](https://alist.hk.tencent.useforall.com/OneDrive%E5%A4%87%E4%BB%BD/Home/%E8%BD%AF%E4%BB%B6%E5%8C%85/TPCLbuild.tar.gz)，并执行下列指令
    ```bash
    tar -zxf TPCLbuild.tar.gz && cd TPCLbuild
    mv tpclash-meta-linux-amd64 /usr/local/bin/tpclash && chmod +x /usr/local/bin/tpclash # 自行根据机器架构选择文件
    mv tpclash.service /etc/systemd/system
    mv clash.yaml /etc/clash.yaml
    sudo systemctl enable tpclash.service --now && systemctl start tpclash.service
    ```
    自行替换clash.yaml的内容

## OpenWRT虚拟机方式
1. 下载[镜像包](https://alist.hk.tencent.useforall.com/OneDrive%E5%A4%87%E4%BB%BD/Home/%E8%BD%AF%E4%BB%B6%E5%8C%85/openwrt-192.168.100.4.gz)并解压后上传至`PVE->local(PVE)->ISO Image`（或者可以自行去[OpenWRT.ai](https://openwrt.ai/)创建自己的镜像）。
2. 创建虚拟机（看图）
    ![CleanShot 2024-10-26 at 01.13.12@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd7309c3767.png)
3. 创建完成后（不要开机虚拟机），进去PVE自身的Shell，import上传的镜像到虚拟机
    ![](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd744274c08.png)
4. 此时回到创建好的虚拟机，点击`Hardware` ，会出现一个未分配磁盘。
    1. 双击磁盘
        ![image.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd74fe0fa74.png)
    2. 点击添加
        ![image2.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd74fbbe340.png)
    3. 扩大空间(不做也可以,不重要)。此时也可以删除`CD/DVD Drive` （不做也可以,不重要）
        ![image3.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd74f5682d7.png)
5. 修改引导顺序，把刚刚设置的硬盘调整到第一位，取消其他的引导盘。记得打开开机自启。最后启动虚拟机。
    ![image4.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd756eebab4.png)
6. 进入OpenWRT（我的镜像默认为192.168.100.4）添加DNS转发指向Clash fakeip端口。
    ![CleanShot 2024-10-26 at 01.24.12@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd75cf64c9e.png)
7. 下载[OpenClash配置备份](https://alist.hk.tencent.useforall.com/OneDrive%E5%A4%87%E4%BB%BD/Home/%E8%BD%AF%E4%BB%B6%E5%8C%85/Backup-OpenClash-Kwrt-x86_64-2024-10-26-00-35-33.tar.gz)，在图片位置上传备份。
    ![cleanshot](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd7659401be.png)
8. 最后回到`运行状态`，往下滑点击`启动OpenClash`。

## 软理由代理设置（这里使用的OPNSense防火墙）
1. 创建网关
    ![OPNSense创建网关](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd76fbc7609.png)
2. `防火墙 → 规则 → LAN`按如图创建规则。（如果需要UDP转发，记得创建第二条的时候把UDP也选上）
    ![CleanShot 2024-10-02 at 12.15.30@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd7762d8080.png)
    
    :::note{title="Docker MACVlan创建"}
    返回`OPNSense → Firewall → Aliases`创建一个别名。可以放上家里其他设备的内网IP在`Content`里。这里只是拿Docker的MacVlan举例。
    ![CleanShot 2024-10-02 at 12.12.49@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd786413204.png)
    还没写，见到请催更😋
    :::
3. `服务 → ISC DHCPv4`。设置分发网关为主路由IP和DNS服务器为OpenWRT IP
    ![CleanShot 2024-10-02 at 12.09.51@2x.png](https://lsky.hk.tencent.useforall.com/Blog/2025/02/25/67bd77b6d8764.png)