https://noviorlu.github.io/categories/ai%E7%AC%94%E8%AE%B0/Recent content in AI笔记 on NoviorluHugo -- gohugo.iozh-CN© 2026 Noviorlu喵Mon, 05 Oct 2026 00:00:00 +0000https://noviorlu.github.io/blog/flashattention-1-to-4/Mon, 05 Oct 2026 00:00:00 +0000https://noviorlu.github.io/blog/flashattention-1-to-4/<style> .fa-intuit{margin:1.4em 0;padding:.85em 1.1em;border-radius:12px;background:rgba(var(--color-primary-500),.08);border-left:3px solid rgb(var(--color-primary-500));line-height:1.65} .fa-intuit>b:first-child{color:rgb(var(--color-primary-600))} html.dark .fa-intuit>b:first-child{color:rgb(var(--color-primary-300))} .fa-road{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:10px;margin:1.6em 0 .6em} @media (max-width:760px){.fa-road{grid-template-columns:repeat(2,minmax(0,1fr))}} .fa-step{border:1px solid rgba(128,128,128,.3);border-radius:12px;padding:.7em .85em;font-size:.82em;line-height:1.5} .fa-step b{display:block;font-size:1.1em} .fa-step i{display:block;font-style:normal;opacity:.65;margin-bottom:.5em} .fa-step span{display:block} .fa-step span+span{margin-top:.45em;color:rgb(var(--color-primary-600))} html.dark .fa-step span+span{color:rgb(var(--color-primary-300))} @media (max-width:640px){.article-content figure{overflow-x:auto}.article-content figure>svg{min-width:520px}} </style> <blockquote><p><strong>Prerequisite:</strong> <a href="https://noviorlu.github.io/blog/gpu-triton-intro/" >GPU 与 Triton 入门</a> (in Chinese) covers the GPU memory hierarchy and Triton basics; <a href="https://noviorlu.github.io/blog/gpu-training-analysis/" >谁偷走了 5090 的算力和显存</a> (in Chinese) covers FLOPs, arithmetic intensity, the roofline model and peak memory, and measures on an RTX 5090 why the $N\times N$ score matrix dominates both the runtime and the activation memory of standard attention — the two problems this post starts from.</p>https://noviorlu.github.io/blog/gpu-training-analysis/Sun, 04 Oct 2026 00:00:00 +0000https://noviorlu.github.io/blog/gpu-training-analysis/<p>我在一张 RTX 5090 上把一步 Transformer 训练拆开，分别量了时间和显存。结果和预想的不太一样：拖慢速度的不是矩阵乘，占显存最多的也不是权重。两边查到最后，都落在 attention 里的两个 seq × seq 矩阵上，一个是分数矩阵 S = QKᵀ（每个 query 对每个 key 的打分），一个是 S 按行过 softmax 之后的注意力权重 P，最后输出是 PV。</p>https://noviorlu.github.io/blog/gpu-triton-intro/Sat, 03 Oct 2026 00:00:00 +0000https://noviorlu.github.io/blog/gpu-triton-intro/<p>这个系列讲怎么让 Transformer 训练在 GPU 上跑得更快。这一篇先打基础：<a href="https://noviorlu.github.io/blog/gpu-triton-intro/#hardware" >第 1 节</a>介绍 GPU 的硬件，包括数据存在哪几层、一个 SM 里有什么、CUDA 的执行层级怎么对应到硬件；<a href="https://noviorlu.github.io/blog/gpu-triton-intro/#triton" >第 2 节</a>用四个例子介绍 Triton，后面两篇的 kernel 都用它写。TPU 和 GPU 的对照放在<a href="https://noviorlu.github.io/blog/gpu-triton-intro/#tpu" >附录 A</a>。</p>https://noviorlu.github.io/blog/openclaw-blog-deployment/Tue, 17 Feb 2026 00:00:00 +0000https://noviorlu.github.io/blog/openclaw-blog-deployment/<h2 class="relative group">起因 <div id="起因" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e8%b5%b7%e5%9b%a0" aria-label="锚点">#</a> </span> </h2> <p>我一直想搭一个个人技术博客，但每次都卡在&quot;选主题 → 改样式 → 写配置 → 部署&quot;的循环里。作为一个图形学方向的程序员，我对前端没什么执念，只想要一个<strong>好看、能用、能写东西</strong>的地方。</p>https://noviorlu.github.io/blog/fundamental-of-rl/Mon, 01 Dec 2025 00:00:00 +0000https://noviorlu.github.io/blog/fundamental-of-rl/<p>得写点东西了，再不写还以为我人消失了 🙂</p> <blockquote><p>原文发布于 <a href="https://zhuanlan.zhihu.com/p/1993276534883119620" target="_blank" rel="noreferrer">知乎</a></p> </blockquote><p><em>内容迁移中&hellip;</em></p>