My Blog Automation Workflow

❗️ Note: it has been days since this article was written, please be aware of its timeliness

A fellow student previously asked me what I use for my blog. I said it’s just Jekyll on GitHub Pages, with the built HTML hosted, but that explanation seemed a bit rough. Plus, the process involves integrating various services, so here I’ll elaborate in detail and share the code.

Overall Workflow

The overall flowchart for Markdown files:

The overall flowchart for blog images:

A few notes on image handling:

During local development, the image repository is a submodule of the blog repository. When adding new images locally, you must first push the images to the image repository, triggering the repository’s action to sync them to Tencent Cloud COS. Only then should you push the blog code; otherwise, the image reference links won’t work.
When writing blog posts, images are referenced using relative paths. Jekyll has three configuration files for local development, .com website references, and .cn website references. The only difference between these YAML files is the static resource references:
Local config file: static_url: /static
.com config file: static_url: https://static.xheldon.com
.cn config file: static_url: https://static.xheldon.cn
To reference image resources in Markdown, the syntax is: \!\[\]\(https://static.xheldon.cn/img/in-post/qing-zheng-lu-yu/IMG_3789.png) (Jekyll processes the Markdown files first, replacing Liquid variables before building the HTML).

Origin

I initially wanted to use Notion as a data source to update the blog, which required a server. So, I purchased a Tencent Cloud Lighthouse server to serve as the backend for fetching Notion data. The results can be seen here:

Subscription & Paid Software - Xheldon Blog

Secondly, another reason is that blogging inevitably requires images. Initially, I used jsDelivr’s Github repository acceleration service:

jsDelivr - A free, fast, and reliable CDN for Open Source

However, the awkward part is that jsDelivr’s acceleration service imposes a 50MB limit per repository, meaning it’s suitable for public js/css files but clearly not ideal for images.

Given this, once I had a server, the solution became obvious: buy another domain and enable CDN acceleration.

Then, taking it a step further—since I’d already bought a domain, and most visitors to my blog are from mainland IPs—why not create a domestic version of the blog? The domain could simply be called https://xheldon.cn.

So, without delay, here’s a summary of the process.

Server and Domain

Purchase

The server I bought is Tencent Cloud’s lightweight server, with 4 cores, 8GB RAM, and 4M bandwidth. It was purchased at a steep discount (think promotional prices at 0.x% off), so it was very affordable.

After buying the domain, of course, it had to be registered for use in mainland China. Without registration, your domain won’t resolve, and accessing it will display messages like “This domain is unregistered and has been suspended.” Fortunately, Tencent Cloud offers free registration services, and the process has been simplified. You just need to fill out Tencent Cloud’s registration application form, providing details like home address, contact information, website purpose, and reasons for application. If your submission doesn’t meet the Ministry of Industry and Information Technology’s requirements—for example, writing something like “Screw the review, just approve it already” in the application reason field—it will definitely be rejected when submitted to the regulatory bureau (the government agency overseeing domain registration is called the “管局”). If there are issues with your submission, a staff member will contact you to help revise and confirm before resubmitting.

Configuration

The configuration involves the following steps, which I’ll briefly outline:

Apply for a free HTTPS certificate and enable HTTPS.
Set up the CDN acceleration domain.
Purchase COS (Cloud Object Storage) for image storage (new users get it for free).
Launching a Node service with Express, exposing port 80 externally via Nginx reverse proxy. The server pulls static HTML resources from Gitee and handles certain API requests. Why not pull directly from GitHub but go through the trouble of these API requests?
Responding to webhooks from Gitee to notify the server to pull the latest HTML files from Gitee.
Handling Notion query requests from the blog. The server will query Notion’s servers to fetch data.

Some points to clarify:

For the lightweight server, I chose a Docker image with Node + Nginx. You could also opt for Java or a custom setup—it’s up to you.
By default, Node and Nginx on the lightweight server are installed under the lighthouse user, often causing permission issues during package installation. To simplify things, I uninstalled the preinstalled Nginx and Node and reinstalled them under the root user.
Occasionally, I need to upload files via FTP to the server, such as the SSL certificates mentioned earlier. This requires configuring FTP-related settings. Tencent Cloud has relevant documentation—just search for it, so I’ll skip the details here.
Domain registration (ICP filing) takes at least a week; in my case, it took about two weeks.

Repository Configuration

Configuring GitHub Actions to Build Static Files

Since I don’t want to expose my blog’s source code and because GitHub Pages has limited support for Jekyll plugins (e.g., pagination on both the homepage and category pages isn’t possible with GitHub Pages’ supported plugins), I decided to build the source code into HTML myself.

Due to the limitations of the free version of GitHub Pages, private repositories cannot enable GitHub Pages. Therefore, I set up another repository as public while keeping the source code repository private. After committing code to the private repository, it is built via GitHub Actions and then pushed to the public repository.

For details on using GitHub Pages and GitHub Actions, you can refer to this article I wrote earlier:

Free Use of Private Repositories to Publish GitHub Pages - Xheldon Blog

However, since I also acquired a domestic domain, some modifications were made to the configuration files. Below are the new configuration files:

name: 博客打包任务

# 代码 push 到 master 分支的时候运行该 workflow
# TODO：不运行 commit 信息中包含特定关键词的 push
on:
  push:
    branches: [ master ]

jobs:
  Build:
    runs-on: ubuntu-latest

    steps:
      - name: 检出分支
        uses: actions/checkout@v2
        with:
          persist-credentials: fasle # false 是用 personal token，true 是使用 GitHub token
          fetch-depth: 0 # 保证能够 push 成功

      # 设置 ruby 环境
      - name: 设置 Ruby 环境
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true
          ruby-version: 2.6

      # 安装依赖
      - name: bundle 安装依赖
        run: bundle install

      # 打包静态资源
      - name: 构建 xheldon.com 页面
        if: ${{ startsWith(github.event.head_commit.message, 'com') || startsWith(github.event.head_commit.message, 'all') }}
        run: bundle exec jekyll build

      - name: xheldon.com 写入信息
        if: ${{ hashFiles('./_site') }}
        working-directory: ./_site
        run: |
            echo "www.xheldon.com" > CNAME
            echo -e "# [Xheldon's blog](https://www.xheldon.com)" > README.md

      - name: 推送到 x_blog 仓库
        if: ${{ hashFiles('./_site') }}
        working-directory: ./_site
        run: |
          pwd
          git init
          git checkout -b master
          git add -A
          git -c user.name='github actions by ${{github.actor}}' -c user.email='NO' commit -m '${{github.event.head_commit.message}}'
          git push "https://${{github.actor}}:${{secrets.X_BLOG_SITE}}@github.com/Xheldon/x_blog.git" HEAD:master -f -q

      - name: 构建 xheldon.cn 页面
        if: ${{ startsWith(github.event.head_commit.message, 'cn') || startsWith(github.event.head_commit.message, 'all') }}
        run: bundle exec jekyll build --config=_config.cn.yml -d _site_cn

      # gitee 和 github 的用户名一样
      - name: 推送到 x_blog_cn 仓库
        if: ${{ hashFiles('./_site_cn') }}
        working-directory: ./_site_cn
        run: |
          pwd
          git init
          git checkout -b master
          git add -A
          git -c user.name='gitub actions by ${{github.actor}} push to gitee' -c user.email='NO' commit -m '${{github.event.head_commit.message}}'
          git push "https://${{github.actor}}:${{secrets.X_BLOG_SITE_CN}}@gitee.com/Xheldon/x_blog_cn.git"  HEAD:master -f -q

Configuring Another Repository to Enable GitHub Pages

This step is self-explanatory—just toggle the GitHub Pages switch for the repository.

Configuring Coding Webhooks

In Gitee’s repository management under Webhooks, configure your WebHook address. Mine is https://www.xheldon.cn/hooks_cn_push。.

Configuring the Server to Respond to Webhooks

The server runs an Express service. Here’s the code directly:

app.post('/hooks_cn_push', async (req, res) => {
    // Note: 收到 x_blog_cn 的 webhooks, 执行 git pull, 将 x_blog_cn 拉取到 public 目录
    // Note: 签名验证，暂时略
    // Note: 拉取分支
    const branch = req.body.repository.clone_url;
    const headers = req.headers;
    console.log('headers:', headers);
    // Note: 验证一下 header 的合法性
    if (
        headers['x-gitee-event'] === 'Push Hook'
            && headers['x-gitee-token'] === GITEE_WEBHOOKS_SECRET
    ) {
        exec(`sudo rm -rf ./_public && sudo git clone ${branch} ./_public && sudo rsync -chir --delete ./_public/ ./public/ && sudo rm -rf ./_public`, {
            cwd: './',

        }, (err, stdout, stderr) => {
            if (err) {
                console.log('err:', err, stderr);
                res.status(400).send({
                    msg: '服务器内部 git clone 仓库失败:',
                    stderr,
                    stdout,
                    err,
                })
            } else {
                // Note: stdout 没有任何输出表示正常
                console.log('out:', stdout);
                if (!stdout) {
                    res.json({
                        msg: '没问题',
                        status: 200,
                        stderr,
                        stdout,
                        err,
                    });
                } else {
                    res.json({
                        msg: 'git clone 的时候返回了一些内容，请过目',
                        status: 200,
                        stderr,
                        stdout,
                        err,
                    });
                }
            }
        });
    } else {
        res.json({
            msg: '恶意请求！',
            status: 403
        });
    }
});

Configuring the Action for the Image Repository

After uploading images to the image repository x_blog-static, a GitHub Action is triggered to incrementally update Tencent Cloud COS. Here’s the code:

name: 腾讯云 cos 同步任务

# 代码 push 到本仓库的时候触发该 ci
# 分析提交的 commits（不只是 head commit），将全部的 added modify 归结到一起（去重）调用上传方法
#  将 delete 的，调用删除方法
on:
  push:
    branches: [master]

jobs:
  CheckOutAndUpload:
    runs-on: ubuntu-latest

    steps:
      - name: 检出分支
        uses: actions/checkout@v2
        with:
          persist-credentials: false
          fetch-depth: 0

      - name: 设置 node 环境
        uses: actions/setup-node@v2.4.1
        with:
          node-version: 14.x
          architecture: x64
          cache: npm

      - name: 安装依赖
        run: npm i

      - name: 运行上传脚本
        uses: actions/github-script@v5
        env:
          COS_SECRET_ID: ${{secrets.COS_SECRET_ID}}
          COS_SECRET_KEY: ${{secrets.COS_SECRET_KEY}}
          COS_BUCKET: ${{secrets.COS_BUCKET}}
          COS_REGION: ${{secrets.COS_REGION}}
        with:
          script: |
            const script = require('./upload.js')
            await script({github, context, core})

Of course, some environment variables, such as COS secrets, can be configured in the repository by referring to the GitHub Action mentioned above.

The above executes a JS file that retrieves commit details to determine what was added, deleted, modified, or renamed in the commit, then performs a batch upload. The repository is public and can be found here.

Future Plans

As mentioned in ](https://www.xheldon.com/tech/use-craft-extension-to-write-blog.html) this article, this post is also synchronized from Craft to a GitHub repository via a plugin. The numerous advantages have already been clearly explained in that article, so I won’t reiterate them here. The only current issue is with images inserted via Craft. Although the official team has released a CORS-unrestricted fetch API (Mac-only), I haven’t yet figured out an elegant way to synchronize images uploaded to Craft with Tencent Cloud COS.

The current plan is to abandon the step of saving images on GitHub and instead directly send images from Craft to Tencent Cloud COS via an extension.

- EOF -

Originally published at: My Blog Automation Workflow - Xheldon Blog