Celeste

Celeste

Short biography for the left-hand sidebar

Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Pages

Posts

[llms] 笔记文档

4 minute read

Published:

（该页面的 .md 和 .pdf 文件已放在视频简介，.md 文件可用该软件 (Obsidian) 或用 VS Code打开）

一、简介

本期视频主要分为以下五部分：

1. 需求和技术
  • 企业对于大模型的不同类型个性化需求
  • SFT（有监督微调）、RLHF（强化学习）、RAG（检索增强生成） -关注：基本概念；分别解决什么问题；如何根据需求选择；
  • 微调部分详细介绍：
    • 微调算法的分类
    • **LoRA 微调算法
    • 微调常见实现框架
      2. 整体步骤说明
  • 在 Linux 系统上微调一个大模型、部署模型、暴露 API 给 web 后端调用，本机前端展示全过程
    3. 模型微调
  • 框架: LLama-Factory (国产最热门的微调框架)
  • 算法: **LoRA (最著名的部分参数微调算法）
  • 基座模型：DeepSeek-R1-Distill-Qwen-1.5B -蒸馏技术通常用于通过将大模型（教师模型）的知识转移到小模型（学生模型）中，使得小模型能够在尽量保持性能的同时，显著减少模型的参数量和计算需求。
    4. 模型部署和暴露接口
  • 框架：FastAPI（一个基于 python 的 web 框架）
    5. web后端调用
  • 通过 HTTP 请求交互即可（ Demo 前后端代码都在视频简介）

[llms] Docs

less than 1 minute read

Published:

个人向学习文档

采样

1 minute read

Published:

sample

  • 采样目的：生成文本时，从预测结果中选出高概率的候选词，避免随机选到低概率词导致的语句不合理，同时保留一定多样性。
  • 先用temperature 调整分布平滑度，然后 top-k + top-p 控制候选范围

MoE 原理及实现

less than 1 minute read

Published:

MoE：mix of experts

  • 专家 stack 起来，计算 token 经过每个专家的输出，对结果加权。

论文阅读: ResNet

1 minute read

Published:

读 ResNet ｜ 图像识别的深度残差学习

如何读论文

less than 1 minute read

Published:

如何读论文

论文阅读: BERT

2 minute read

Published:

读 BERT ｜ 针对语言理解任务预训练的深度双向 Transformer

论文阅读: CCV

less than 1 minute read

Published:

读 CCV ｜ 用于情景医学图像分割的循环上下文验证

LoRA 原理及实现

1 minute read

Published:

LoRA：low rank adaptation

  • 用两个小矩阵相乘，去表示全量微调后权重矩阵的这个变化量

论文阅读: GPT-2.0

1 minute read

Published:

读 GPT-2.0 ｜ 语言模型是无监督的多任务学习

论文阅读: DeepSeek-R1

2 minute read

Published:

读 DeepSeek-R1 ｜ 通过强化学习激励大语言模型的推理能力

论文阅读: GPT-1.0

22 minute read

Published:

读 GPT-1.0 ｜ 通过生成式预训练来提高语言理解

LLMs: Seq2seq

5 minute read

Published:

Sequence to Sequence Learning with Neural Networks

cs231n_cnn_3

less than 1 minute read

Published:

CS231n_cnn: 3. Transfer Learning

cs231n_cnn_2

less than 1 minute read

Published:

CS231n_cnn: 2. Visualizing what ConvNets learn

cs231n_cnn_1

7 minute read

Published:

CS231n_cnn: 1. Convolutional Neural Network

cs231n_8

3 minute read

Published:

CS231n: 8. Minimal Neural Network Case

cs231n_7

6 minute read

Published:

CS231n: 7. Learning the parameters

cs231n_6

5 minute read

Published:

CS231n: 6. Setting up the data and the model

cs231n_5

3 minute read

Published:

CS231n: 5. Architecture, ReLU, overfitting

cs231n_4

2 minute read

Published:

CS231n: 4. Backpropagation

cs231n_3

2 minute read

Published:

CS231n: 3. Optimization: Stochastic Gradient Descent

cs231n_2

3 minute read

Published:

CS231n: 2. 线性分类器, SVM loss, Softmax

cs231n_1

2 minute read

Published:

CS231n: 1. Introduction & KNN & Data Split

卷积计算

1 minute read

Published:

卷积计算

portfolio

publications

Paper Title Number 4

Published in GitHub Journal of Bugs, 2024

This paper is about fixing template issue #693.

Recommended citation: Your Name, You. (2024). "Paper Title Number 3." GitHub Journal of Bugs. 1(3).
Download Paper

talks

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.