目录整体架构总览核心类与结构体整体调用链路一、两种实现子类详解1. LscTable 网格查表模式type: table关键行为2. LscPolynomial 径向多项式模式type: polynomial多项式核心计算二、上层控制逻辑 lsc.cpp/lsc.hLscAlgorithmBaseLscAlgorithm 模板类configure () 核心函数interpolateComponents(unsigned int ct)getComponents()三、Table 模式 vs Polynomial 模式对比四、完整调用时序五、工程踩坑清单六、代码实现整体架构总览采用虚基类 多实现的设计模式将调参解析、增益生成与上层控制、色温插值、定点量化做解耦。核心类与结构体LscImplementationlsc_base.h纯虚基类定义统一接口隔离两种 LSC 实现。parseLscData()解析 tuning 配置文件加载多色温 LSC 数据集。sampleForCrop()根据 sensor 的 analogCrop重采样生成适配当前画面的增益集合。两个别名Componentsstd::mapstd::string, std::vectorfloatkey 为颜色通道名字value 为通道对应的网格增益数组。注意Table 和 Polynomial 模式下vectorfloat存储的数值语义完全不一样。ComponentsMapstd::mapunsigned int, Componentskey 为色温 ct存储多套不同色温的 LSC 数据。派生实现LscTable网格查表模式不支持 crop 重采样源码标记 TODO 待实现。LscPolynomial径向多项式模式支持任意 analogCrop、任意输出网格遵循 Adobe DNG FixVignetteRadial 晕圈模型。LscAlgorithmBase非模板上层基类负责 tuning 初始化、处理 APP 下发的 LSC 开关控制、metadata 元数据回填。LscAlgorithmU模板子类lsc.h模板参数U为平台定点量化工具类负责浮点增益向 ISP 硬件定点格式转换 内部持有InterpolatorComponents sets_提供色温线性插值接口输出可直接下发 ISP 的量化增益表。状态结构体ActiveState跨帧持久状态仅成员bool enabled保存 LSC 全局使能标记。FrameContext单帧独立上下文每帧重新生成enabled本帧 LSC 是否开启update标记本帧是否需要更新 LSC 参数。LscDescriptor平台描述符平台 IPA 必须填充struct LscDescriptor { std::vectorstd::string keys; // 通道名称集合 {r,gr,gb,b} 或者 {r,g,b} unsigned int numHSamples; // ISP LSC网格水平采样点数 unsigned int numVSamples; // ISP LSC网格垂直采样点数 Size sensorSize; // 完整原始sensor像素尺寸多项式模式必需 };Table 模式使用numHSamples * numVSamples校验 yaml 增益数组长度。Polynomial 模式sensorSize用于多项式的坐标归一化与 M 值计算。整体调用链路plaintextinit() → 根据tuning中type字段创建LscTable/LscPolynomial实例解析sets多色温数据集 queueRequest() → 处理Request携带的LensShadingCorrectionEnable控制更新持久态、标记帧上下文update configure() → 调用impl_-sampleForCrop()完成crop重采样将增益量化为ISP定点格式存入sets_插值容器 interpolateComponents(ct) → 根据AWB输出色温在多色温数据集之间做线性插值输出最终增益集合 process() → 将本帧LSC使能状态写入metadata元数据重要LSC 算法模块不操作 ISP 硬件寄存器仅输出增益数据集由上层平台 IPA 负责把增益写入 ISP。一、两种实现子类详解1. LscTable 网格查表模式type: tabletuning yaml 中直接存储每个色温下各通道完整的网格增益数组。yaml- Lsc: type: table sets: - ct: 2500 r: [定点编码值0, 定点编码值1, ...] g: [定点编码值0, 定点编码值1, ...] b: [定点编码值0, 定点编码值1, ...]关键行为parseLscData()遍历 sets 数组校验每个通道数组元素个数等于numHSamples * numVSamples yaml 中的数值是ISP 定点寄存器字面量读取存入std::vectorfloat只是容器类型为 float没有做浮点增益换算。源码 TODO未来希望 table 模式 tuning 直接存储物理浮点增益统一在 configure 做量化。sampleForCrop()打印警告日志不执行重采样直接返回原始加载的数据集。限制当 sensor 开启 analog crop、binning输出网格不会跟随画面变化LSC 校正效果失效。⚠️语义重点Table 模式的Components内部vectorfloat存储的是ISP 定点编码不是物理增益1.0 不代表无校正。2. LscPolynomial 径向多项式模式type: polynomial遵循 Adobe DNG FixVignetteRadial 径向晕圈模型每个颜色通道存储一组参数光学中心cx, cy多项式系数k0~k4。yaml- Lsc: type: polynomial sets: - ct: 2500 r: {cx:0.500, cy:0.510, k0:1.539, k1:-1.143, k2:4.332, k3:0, k4:0} gr: {...} gb: {...} b: {...}多项式核心计算M 值计算getM()M原始 sensor 尺寸下光学中心到图像最远角点的欧几里得像素距离。注意cx_、cy_是调参得到的相对于完整 sensor 的归一化光学中心不一定等于图像几何中心。double cpx imageSize_.width * cx_; double cpy imageSize_.height * cy_; double mx std::max(cpx, std::fabs(imageSize_.width - cpx)); double my std::max(cpy, std::fabs(imageSize_.height - cpy)); return sqrt(mx * mx my * my);坐标归一化所有像素坐标全部除以 M得到 DNG 规范定义的归一化坐标系。增益采样函数sampleAtNormalizedPixelPos(x,y)输入为除以 M 后的归一化坐标 xp、ypdouble dx x - cnx_; double dy y - cny_; double r sqrt(dx * dx dy * dy); double res 1.0; for (unsigned int i 0; i coefficients_.size(); i) res coefficients_[i] * std::pow(r, (i 1) * 2); return res;公式返回值为物理浮点增益1.0代表无需校正补偿。r除以 M 后的归一化径向距离不是原始像素距离也不是普通 0‑1 图像归一化坐标。samplePolynomial()两层坐标映射xPos/yPos相对于 crop 区域的 [0‑1] 网格顶点坐标。double x0 cropRectangle.x / m; double y0 cropRectangle.y / m; double w cropRectangle.width / m; double h cropRectangle.height / m; double xp x0 x * w; double yp y0 y * h;变换链路crop 局部归一坐标 → DNG M 归一化坐标系。 采样输出顺序y 外层循环、x 内层循环行优先输出数组排布必须和 ISP 硬件网格加载顺序一致否则颜色错乱。sampleForCrop()遍历全部色温集合对每个通道多项式按传入的网格节点采样输出物理浮点增益集合。✅支持任意 analogCrop任意输出网格。⚠️注意调参的cx/cy是相对于完整 sensor 尺寸不是 crop 后的画面。LscDescriptor::sensorSize必须和标定时使用的 sensor 尺寸完全一致。二、上层控制逻辑 lsc.cpp/lsc.hLscAlgorithmBaseinit()读取 tuning 中type字段实例化对应实现类LscTable/LscPolynomial调用parseLscData解析 sets 数据集注册控制项LensShadingCorrectionEnable。queueRequest()处理 Request 携带的LensShadingCorrectionEnable控制。如果使能状态发生改变更新持久态ActiveState::enabled标记FrameContext::update true将状态同步至本帧上下文FrameContext::enabled。process()将本帧的context.enabled写入 metadata 元数据回传给上层应用。LscAlgorithm模板类configure () 核心函数//1. 根据当前analogCrop重采样得到浮点ComponentsMap LscImplementation::ComponentsMap data impl_-sampleForCrop(analogCrop, xPos, yPos); //2. 做定点转换存入插值容器sets_ for (auto [t, c] : data) { for (auto [k, gains] : c) { for(auto gain : gains) { if (polynomial_) //多项式物理浮点增益 → ISP定点格式执行量化运算 quantizedGains.push_back(U(gain).quantized()); else //table模式已经是定点字面量仅做隐式类型转换不做量化 quantizedGains.push_back(gain); } } } sets_.setData(std::move(lscData)); state.enabled true;关键差异Polynomial输出物理浮点增益运行时执行浮点→定点量化。Table数据来自 yaml 的定点字面量只做类型转换不做量化运算。interpolateComponents(unsigned int ct)const Components interpolateComponents(unsigned int ct) { return sets_.getInterpolated(ct); }输入 AWB 输出的色温 ct调用Interpolator工具在已加载的多色温数据集之间做线性插值。底层模板特化实现对每个颜色通道的增益 vector 逐元素线性插值dest[i] a[i]*(1‑lambda)b[i]*lambda。插值的 clamp 限幅行为来自通用Interpolator组件不属于 LSC 模块内部逻辑。约束调用该接口前必须先调用configure()否则 sets_为空返回空数据集。getComponents()返回全部未插值的原始多色温数据集用于调试。三、Table 模式 vs Polynomial 模式对比项目LscTable 网格查表模式LscPolynomial 径向多项式模式调参存储每个色温存储完整二维增益网格数组yaml 存放 ISP 定点字面量每个通道存储多项式系数 cx,cy,k0~k4Components 内 vectorfloat语义存储ISP 定点寄存器编码值不是物理增益存储物理浮点增益1.0 代表无校正analogCrop 重采样❌不支持直接返回原始表并打印警告✅支持任意 crop、任意输出网格节点configure 阶段处理仅做数值隐式转换不执行浮点‑定点量化采样输出浮点增益调用 U 执行浮点→定点量化调参可移植性绑定 ISP 定点格式不能跨平台直接复用纯浮点参数与 ISP 硬件无关可跨平台复用数据体积大保存多套完整网格极小仅保存多项式系数依赖配置项numHSamples、numVSamplesLscDescriptor::sensorSize完整 sensor 像素尺寸四、完整调用时序IPA 初始化实例化LscAlgorithmU填充平台LscDescriptor描述符调用init(tuningData, controls, descriptor)解析 yaml 配置实例化对应的 LSC 实现流媒体启动 /sensor 切换分辨率 /analogCrop调用configure(state, analogCrop, xPos, yPos)polynomial 模式多项式按 crop 与网格节点采样得到物理浮点增益量化为 ISP 定点存入sets_table 模式返回原始数据集仅做类型转换帧循环queueRequest(state, context, request.controls)处理 APP 下发 LSC 开关修改持久态ActiveState填充本帧FrameContext上层 IPA 拿到 AWB 输出色温 ct调用interpolateComponents(ct)得到插值完成的量化增益集合平台 IPA 负责将增益集合下发 ISP 硬件process(context, metadata)将本帧 LSC 使能状态写入 metadata 元数据。五、工程踩坑清单LscDescriptor::keys必须与 yaml 中每个 set 下的通道 key 严格匹配区分r/gr/gb/b否则解析返回‑EINVAL。Table 模式下sensor 使用 analog‑crop /binningLSC 网格不会跟随画面变化校正失效项目条件允许优先使用 Polynomial 模式。Polynomial 模式LscDescriptor::sensorSize必须等于标定多项式时的完整 sensor 尺寸填错会造成坐标变换全部错误。configure()必须在interpolateComponents()之前调用未配置时 sets_为空输出空的增益集合。Polynomial 调参中cx、cy是完整 sensor 的归一化光学中心不是 crop 之后画面的坐标。不要混用两种模式的增益语义Table 的 yaml 数值是寄存器定点编码Polynomial 输出 1.0 代表不需要亮度补偿。Polynomial 模式输出数组顺序y 外层循环、x 内层循环行优先输出顺序必须和 ISP 硬件网格加载顺序保持一致否则画面颜色错乱。色温插值的越界限幅是通用Interpolator组件行为不是 LSC 模块实现。插值时要求两套参与插值的 Components 的 key 集合、每个 vector 的长度完全一致否则触发 ASSERT 崩溃该条件由 configure 流程保证。六、代码实现/* SPDX-License-Identifier: LGPL-2.1-or-later */ /* * Copyright (C) 2024, Ideas On Board * * Polynomial based lens shading correction */ #include lsc_polynomial.h #include assert.h #include cmath #include libcamera/base/log.h /** * \file lsc_polynomial.h * \brief LscPolynomial class */ namespace libcamera { LOG_DEFINE_CATEGORY(LscPolynomial) namespace ipa { namespace lsc { /** * \class Polynomial * \brief Class for handling even polynomials used in lens shading correction * * Shading artifacts of camera lenses can be modeled using even radial * polynomials. This class implements a polynomial with 5 coefficients which * follows the definition of the FixVignetteRadial opcode in the Adobe DNG * specification. */ /** * \fn Polynomial::Polynomial(double cx 0.0, double cy 0.0, double k0 0.0, double k1 0.0, double k2 0.0, double k3 0.0, double k4 0.0) * \brief Construct a polynomial using the given coefficients * \param cx Center-x relative to the image in normalized coordinates (0..1) * \param cy Center-y relative to the image in normalized coordinates (0..1) * \param k0 Coefficient of the polynomial * \param k1 Coefficient of the polynomial * \param k2 Coefficient of the polynomial * \param k3 Coefficient of the polynomial * \param k4 Coefficient of the polynomial */ /** * \brief Sample the polynomial at the given normalized pixel position * * This functions samples the polynomial at the given pixel position divided by * the value returned by getM(). * * \param x x position in normalized coordinates * \param y y position in normalized coordinates * \return The sampled value */ double Polynomial::sampleAtNormalizedPixelPos(double x, double y) const { double dx x - cnx_; double dy y - cny_; double r sqrt(dx * dx dy * dy); double res 1.0; for (unsigned int i 0; i coefficients_.size(); i) res coefficients_[i] * std::pow(r, (i 1) * 2); return res; } /** * \brief Get the value m as described in the dng specification * * Returns m according to dng spec. m represents the Euclidean distance * (in pixels) from the optical center to the farthest pixel in the * image. * * \return The sampled value */ double Polynomial::getM() const { double cpx imageSize_.width * cx_; double cpy imageSize_.height * cy_; double mx std::max(cpx, std::fabs(imageSize_.width - cpx)); double my std::max(cpy, std::fabs(imageSize_.height - cpy)); return sqrt(mx * mx my * my); } /** * \brief Set the reference image size * * Set the reference image size that is used for subsequent calls to getM() and * sampleAtNormalizedPixelPos() * * \param size The size of the reference image */ void Polynomial::setReferenceImageSize(const Size size) { assert(!size.isNull()); imageSize_ size; /* Calculate normalized centers */ double m getM(); cnx_ (size.width * cx_) / m; cny_ (size.height * cy_) / m; } } /* namespace lsc */ /** * \class LscPolynomial * \brief Radial Polynomial LSC algorithm implementation * * Polynomial-based LSC algorithm implementation. The LscPolynomial class * implements LSC support using a Polynomial to represent the shading artifacts * map. * * \sa LscImplementation */ /** * \brief Parse polynomial LSC data * \param[in] sets The tuning file content * \param[in] descriptor The LSC engine descriptor * * Parse the LSC data in polyomial form from the \a sets tuning data. * * \return 0 on success or a negative error number otherwise */ int LscPolynomial::parseLscData(const ValueNode sets, const LscDescriptor descriptor) { for (const auto set : sets.asList()) { uint32_t ct set[ct].getuint32_t(0); PolynomialComponents components; for (auto k : descriptor.keys) { auto polynomial set[k].getlsc::Polynomial(); if (!polynomial) { LOG(LscPolynomial, Error) Missing polynomial for component k; return -EINVAL; } auto [it, inserted] components.try_emplace(k, std::move(*polynomial)); ASSERT(inserted); it-second.setReferenceImageSize(descriptor.sensorSize); } auto [it, inserted] lscData_.try_emplace(ct, std::move(components)); if (!inserted) { LOG(LscPolynomial, Error) Multiple sets found for color temperature ct; return -EINVAL; } } if (lscData_.empty()) { LOG(LscPolynomial, Error) Failed to load any sets; return -EINVAL; } return 0; } /** * \brief Re-sample the LSC components for \a cropRectangle * \param[in] cropRectangle The sensor analogue crop rectangle * \param[in] xPos List of horizontal positions of the LSC grid nodes * \param[in] yPos List of vertical positions of the LSC grid nodes * * LSC tables have to be re-sampled every time a new sensor configuration is * used, as each streaming session might use a different sensor crop rectangle. * * Polynomial LSC tables can be re-sampled for a given sensor frame resolution * using a list of horizontal and vertical nodes that define the LSC grid on * which the polynomial is re-sampled on. * * \a cropRectangle represents the size of the frame on which the LSC tables * have to be re-sampled on. * * \a xPos and \a yPos represent the position of the grid nodes vertexes in * the [0, 1] interval. In example an equally spaced grid of 16 nodes will have * each segment of size 0.0625 and the list of nodes position will be * [0, 0.0625, 0.125, 0.1875, ... , 1]. It is expected that the first position * is 0 and the last position is 1. */ LscImplementation::ComponentsMap LscPolynomial::sampleForCrop(const Rectangle cropRectangle, std::vectordouble xPos, std::vectordouble yPos) { LscImplementation::ComponentsMap components; for (const auto [t, c] : lscData_) { LscImplementation::Components comp components[t]; for (const auto [k, p] : c) comp.try_emplace(k, samplePolynomial(p, xPos, yPos, cropRectangle)); } return components; } std::vectorfloat LscPolynomial::samplePolynomial(const lsc::Polynomial poly, Spanconst double xPositions, Spanconst double yPositions, const Rectangle cropRectangle) { double m poly.getM(); double x0 cropRectangle.x / m; double y0 cropRectangle.y / m; double w cropRectangle.width / m; double h cropRectangle.height / m; std::vectorfloat samples; samples.reserve(xPositions.size() * yPositions.size()); for (double y : yPositions) { for (double x : xPositions) { double xp x0 x * w; double yp y0 y * h; samples.push_back(static_castfloat (poly.sampleAtNormalizedPixelPos(xp, yp))); } } return samples; } } /* namespace ipa */ #ifndef __DOXYGEN__ template std::optionalipa::lsc::Polynomial ValueNode::Accessoripa::lsc::Polynomial::get(const ValueNode obj) const { std::optionaldouble cx obj[cx].getdouble(); std::optionaldouble cy obj[cy].getdouble(); std::optionaldouble k0 obj[k0].getdouble(); std::optionaldouble k1 obj[k1].getdouble(); std::optionaldouble k2 obj[k2].getdouble(); std::optionaldouble k3 obj[k3].getdouble(); std::optionaldouble k4 obj[k4].getdouble(); if (!(cx cy k0 k1 k2 k3 k4)) { LOG(LscPolynomial, Error) Polynomial is missing a parameter; return std::nullopt; } return ipa::lsc::Polynomial(*cx, *cy, *k0, *k1, *k2, *k3, *k4); } #endif /* __DOXYGEN__ */ } /* namespace libcamera */