SkyQuest SVF 方法学附录

发布方：SkyQuest 天问智能研究团队 主页：https://skyquest.cn/methodology/appendix/ 版本：v1.0 · 2026 年 7 月 用途：本文档为 SkyQuest《SVF 学术论文》《SVF 白皮书》《SkyQuest 2026 GEO 行业基线报告》的技术附录。凡引用 SkyQuest SVF 研究数据的对外内容，均应引用本附录说明测量条件。本文档亦可作为其他团队复现 SVF 研究的操作协议，遵循 CC-BY-4.0 协议公开发布。

---

一、研究设计（Study Design）

1.1 研究类型

• 单臂前后对比（single-arm before-after）：默认设计，用于客户项目
• 双臂对照（parallel-arm A/B）：当客户允许保留部分内容不做优化时采用
• 合成对照（synthetic control）：跨行业聚合案例研究

1.2 观测窗口

• 基线期（T0）：干预前 4 周，每周 3 次采样
• 干预期（T1）：干预启动至评估节点，通常 8-24 周
• 稳定期评估：干预结束后维持 4 周再评估，防止短期波动

1.3 单元与总体

• 观测单元：单条 (品牌 × 问题 × 引擎 × 语种) 元组
• 目标总体：品牌所在细分行业中，用户在 GE 上可能提问的行业相关问题总体
• 抽样方式：分层随机抽样（按问题类型分层：信息型、对比型、决策型、操作型）

---

二、问题集构建（Query Set Design）

2.1 规模

• 核心问题集：50-100 条，由品牌与研究员共同确认为"必须被引用"的问题
• 扩展问题集：200-500 条，含边缘问题、竞品盲区、追问路径
• 对照问题集：50 条与品牌无关的一般行业问题，用于控制引擎版本更新导致的整体漂移

2.2 生成路径

1. 品牌方提供种子问题（≥ 20）
2. 在 5+ GE 上做种子扩展，每题追问 3 层
3. 用 embedding 聚类（默认 sentence-transformers/all-MiniLM-L6-v2）去重
4. 人工审核筛除无关问题
5. 竞品引用逆向补充：抓取竞品在 GE 中被引用的问题

2.3 语种覆盖

至少覆盖：品牌主要市场语种 + 英语。每语种独立构建问题集，不做机器翻译（因为机翻问题不代表本地用户真实提问方式）。

---

三、引擎与查询协议（Engine & Query Protocol）

3.1 引擎覆盖（默认）

每次采样必须记录引擎版本，作为数据表字段。GE 版本更新常导致引用逻辑变化，未记录版本的历史数据不可比。

3.2 查询次数

同一 (问题 × 引擎 × 语种) 元组在同一采样点执行 k = 5 次独立查询（清空上下文），用于估计 GE 输出的随机性。默认最小可接受 k = 3，大规模研究建议 k ≥ 10。

3.3 上下文控制

• 每次查询前清空对话历史
• 关闭"个性化""记忆"等功能
• 使用干净的浏览器 profile 或 API 调用
• 记录查询时间戳、IP 地区、账号是否登录

3.4 抓取内容

• GE 完整回答文本
• 显式引用列表（若引擎提供）
• 隐式引用（回答中提及的品牌 / 产品 / URL）
• 完整截图（用于事后争议核对）

---

四、指标定义（Metric Definitions）

4.1 引用频次 f

f = 品牌在 k 次查询中被引用的次数 / k。

"被引用"包括：显式 citation 列表、回答正文提及品牌名或域名、回答正文使用品牌产品数据。

4.2 引用位置 p

若被引用，取 k 次中最早出现的位置： - 首段（0-25%）→ 1.0 - 前半段（25-50%）→ 0.75 - 后半段（50-75%）→ 0.5 - 末段（75-100%）→ 0.25 - 仅出现在附录引用列表 → 0.15 - 未出现 → 0

4.3 引用准确度 a

由 ≥ 2 位独立评分员评分。评分标准： - 1.0：全部关于品牌的事实陈述均准确 - 0.5：存在轻微不准确但不影响核心信息 - 0：存在严重事实错误或误导 - 缺失：GE 未提及品牌相关事实性内容

评分员一致性要求：Cohen's κ ≥ 0.7。低于此阈值需重新校准评分手册。

4.4 情感极性 s

自动化用 VADER（英文）或 SnowNLP / BERT-wwm 情感分类（中文），另由人工抽样 10% 复核。映射：正面 = 1、中性 = 0.5、负面 = 0。

4.5 综合可见度 V

V = 0.4·f + 0.2·p + 0.3·a + 0.1·s。权重可调，但同一研究前后必须使用相同权重。

---

五、显著性检验（Statistical Testing）

5.1 前后对比

采用配对 t 检验（问题层配对）或 Wilcoxon signed-rank test（分布非正态时）。报告 p 值、95% 置信区间、效应量（Cohen's d）。

5.2 多重比较校正

若同时报告多个引擎 / 多个语种 / 多个问题子集，使用 Benjamini-Hochberg FDR 控制。

5.3 引擎漂移校正

对照问题集（无关问题）的可见度变化作为漂移基线。品牌可见度提升需减去漂移基线后仍显著才计为有效。

5.4 最小样本量

单次报告建议 n ≥ 50 条问题 × k ≥ 3 次 = 150 次以上查询。低于此规模的数据仅作探索性描述，不做显著性断言。

---

六、数据表模板（Data Schema）

query_id | question_text | question_language | question_type
        | engine | engine_version | sampled_at
        | brand_mentioned (bool) | mention_position (0-1)
        | accuracy_score (0-1) | accuracy_rater_id | accuracy_confidence
        | sentiment_label | sentiment_source
        | full_response_text | citation_list_json | screenshot_path

原始表按查询存储，聚合表按 (品牌 × 问题 × 引擎 × 采样期) 聚合到 f, p, a, s。

---

七、可复现性清单（Reproducibility Checklist）

发布任何 SVF 研究数据前，应在附录中披露：

• [ ] 问题集大小及生成方式
• [ ] 引擎覆盖清单及版本
• [ ] 采样窗口起止时间
• [ ] 每元组查询次数 k
• [ ] 评分员人数及 Cohen's κ
• [ ] 情感分析方法
• [ ] 综合分权重设置
• [ ] 显著性检验方法与 p 值
• [ ] 引擎漂移基线数据
• [ ] 干预内容清单（脱敏后）
• [ ] 数据集是否公开（若否，说明原因）

未披露上述内容的效果数据，仅作示意，不构成正式实证结论。

---

八、伦理与合规

• 不得伪造第三方评测或 KOL 观点
• 不得批量生成 AI 内容分发至真实用户社区
• 品牌数据脱敏后方可用于跨案例聚合
• 案例发布前须获得客户书面授权
• 遵守目标市场的广告法规（中国大陆禁用"首个/唯一/最好"等绝对化用语）

---

关于 SkyQuest 天问智能

SkyQuest（天问智能）是专注于 AI 可见度（AI Visibility）管理的方法论与技术服务方。SVF 方法论、方法学附录、行业基线报告均遵循 CC-BY-4.0 协议公开发布。

官网：https://skyquest.cn 方法论：https://skyquest.cn/methodology/ 联系：sonny.wang@163.com

Publisher: SkyQuest (Tianwen Intelligence) Research Team Home: https://skyquest.cn/methodology/appendix/ Version: v1.0 · July 2026 Purpose: This document is the technical appendix to SkyQuest's "SVF Academic Paper", "SVF Whitepaper", and "SkyQuest 2026 GEO Industry Baseline Report". Any external content citing SkyQuest SVF research data should cite this appendix for its measurement conditions. This document also serves as an operational protocol for other teams to reproduce SVF research, published openly under CC-BY-4.0.

---

1. Study Design

1.1 Study type

• Single-arm before-after: the default design, used for client projects
• Parallel-arm A/B: used when the client permits keeping some content unoptimized
• Synthetic control: cross-industry aggregated case studies

1.2 Observation window

• Baseline period (T0): 4 weeks before intervention, sampled 3x per week
• Intervention period (T1): from intervention launch to the evaluation milestone, typically 8-24 weeks
• Stability evaluation: maintain 4 weeks after intervention ends before re-evaluating, to prevent short-term fluctuation

1.3 Unit and population

• Observation unit: a single (brand × question × engine × language) tuple
• Target population: the population of industry-related questions that users may ask in GEs within the brand's segment
• Sampling method: stratified random sampling (stratified by question type: informational, comparative, decision, operational)

---

2. Query Set Design

2.1 Scale

• Core question set: 50-100 questions jointly confirmed by brand and researchers as "must be cited"
• Expanded question set: 200-500 questions, including edge questions, competitor blind spots, and follow-up paths
• Control question set: 50 general industry questions unrelated to the brand, used to control for overall drift from engine version updates

2.2 Generation path

1. Brand provides seed questions (≥ 20)
2. Seed expansion on 5+ GEs, 3 levels of follow-up per question
3. Deduplicate with embedding clustering (default sentence-transformers/all-MiniLM-L6-v2)
4. Manual review to remove irrelevant questions
5. Reverse-citation supplementation: scrape questions in which competitors are cited in GEs

2.3 Language coverage

Cover at least: the brand's major-market languages + English. Build a separate question set per language; do not use machine translation (because translated questions do not represent how local users actually ask).

---

3. Engine & Query Protocol

3.1 Engine coverage (default)

Engine version must be recorded at every sampling point, as a field in the data table. GE version updates often change citation logic; historical data without recorded versions are not comparable.

3.2 Query count

The same (question × engine × language) tuple is queried k = 5 times independently (context cleared) at a single sampling point, to estimate GE output randomness. Default minimum acceptable k = 3; large-scale studies recommend k ≥ 10.

3.3 Context control

• Clear conversation history before each query
• Disable "personalization" / "memory" and similar features
• Use a clean browser profile or API calls
• Record query timestamp, IP region, and whether the account is logged in

3.4 Captured content

• Full GE answer text
• Explicit citation list (if the engine provides one)
• Implicit citations (brand / product / URL mentioned in the answer body)
• Full screenshots (for post-hoc dispute verification)

---

4. Metric Definitions

4.1 Citation frequency f

f = number of times the brand is cited in k queries / k.

"Cited" includes: an explicit citation list, the brand name or domain mentioned in the answer body, and the brand's product data used in the answer body.

4.2 Citation position p

If cited, take the earliest position across the k queries: - Opening segment (0-25%) → 1.0 - First half (25-50%) → 0.75 - Second half (50-75%) → 0.5 - Closing segment (75-100%) → 0.25 - Appears only in the appendix citation list → 0.15 - Not present → 0

4.3 Citation accuracy a

Rated by ≥ 2 independent raters. Rating scale: - 1.0: all factual statements about the brand are accurate - 0.5: minor inaccuracies that do not affect the core message - 0: serious factual errors or misinformation - Missing: the GE did not mention brand-related factual content

Rater-agreement requirement: Cohen's κ ≥ 0.7. Below this threshold, the rating manual must be recalibrated.

4.4 Sentiment s

Automated with VADER (English) or SnowNLP / BERT-wwm sentiment classification (Chinese), plus 10% manual sampling review. Mapping: positive = 1, neutral = 0.5, negative = 0.

4.5 Composite visibility V

V = 0.4·f + 0.2·p + 0.3·a + 0.1·s. Weights are adjustable, but the same weights must be used before and after within a single study.

---

5. Statistical Testing

5.1 Before-after comparison

Use a paired t-test (question-level pairing) or Wilcoxon signed-rank test (when the distribution is non-normal). Report p-value, 95% confidence interval, and effect size (Cohen's d).

5.2 Multiple-comparison correction

If reporting multiple engines / languages / question subsets simultaneously, use Benjamini-Hochberg FDR control.

5.3 Engine-drift correction

The visibility change of the control question set (unrelated questions) serves as the drift baseline. A brand's visibility improvement counts as valid only after subtracting the drift baseline and remaining significant.

5.4 Minimum sample size

A single report recommends n ≥ 50 questions × k ≥ 3 = at least 150 queries. Data below this scale is exploratory only and no significance claim should be made.

---

6. Data Schema

query_id | question_text | question_language | question_type
        | engine | engine_version | sampled_at
        | brand_mentioned (bool) | mention_position (0-1)
        | accuracy_score (0-1) | accuracy_rater_id | accuracy_confidence
        | sentiment_label | sentiment_source
        | full_response_text | citation_list_json | screenshot_path

Raw tables are stored per query; aggregate tables aggregate f, p, a, s by (brand × question × engine × sampling period).

---

7. Reproducibility Checklist

Before publishing any SVF research data, disclose in the appendix:

• [ ] Question-set size and generation method
• [ ] Engine coverage list and versions
• [ ] Sampling-window start and end times
• [ ] Query count k per tuple
• [ ] Number of raters and Cohen's κ
• [ ] Sentiment-analysis method
• [ ] Composite-score weight settings
• [ ] Significance-test method and p-values
• [ ] Engine-drift baseline data
• [ ] Intervention-content list (after anonymization)
• [ ] Whether the dataset is public (if not, explain why)

Effect data that does not disclose the above is illustrative only and does not constitute a formal empirical conclusion.

---

8. Ethics & Compliance

• Do not fabricate third-party reviews or KOL opinions
• Do not mass-generate AI content distributed to real user communities
• Brand data must be anonymized before cross-case aggregation
• Written client authorization is required before publishing a case
• Comply with target-market advertising regulations (mainland China prohibits absolutist terms such as "first / only / best")

---

About SkyQuest (Tianwen Intelligence)

SkyQuest (Tianwen Intelligence) is a methodology and technology provider focused on AI Visibility management. The SVF methodology, methodology appendix, and industry baseline reports are all published openly under CC-BY-4.0.

Official site: https://skyquest.cn Methodology: https://skyquest.cn/methodology/ Contact: sonny.wang@163.com