ruoyunbai/compliance
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

调试网络

Browse Source
This commit is contained in:
gongwenxin 2025-08-07 15:23:25 +08:00
parent fa343eb111
commit 53aa2a647b
10 changed files with 602 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -4,6 +4,7 @@
./memory-bank
./logs
./build
log*
# Python相关
__pycache__/

24
README.md
View File

@ -41,11 +41,19 @@ python history_viewer.py
### 命令行测试
```bash
# 基本使用
python run_api_tests.py \
--base-url http://your-api-server \
--openapi-file api-spec.yaml \
--strictness-level HIGH \
--generate-pdf
# DMS服务测试（忽略SSL证书）
python run_api_tests.py \
--dms ./assets/doc/dms/domain.json \
--base-url https://www.dev.ideas.cnpc \
--ignore-ssl \
--strictness-level CRITICAL
```
### Web界面测试
@ -103,6 +111,22 @@ python tests/test_multi_service.py
## 🛠️ 故障排除
### SSL证书问题
如果遇到SSL证书验证失败：
```bash
# 使用--ignore-ssl参数（开发/测试环境）
python run_api_tests.py --dms ./assets/doc/dms/domain.json --ignore-ssl
# 测试SSL配置
python test_ssl_ignore.py
```
详细说明请查看：[SSL证书处理指南](docs/SSL_Certificate_Handling.md)
### 其他问题
```bash
# 查看容器状态
docker ps

BIN
ddms_compliance_suite/__pycache__/test_orchestrator.cpython-312.pyc
View File

Binary file not shown.

BIN
ddms_compliance_suite/input_parser/__pycache__/parser.cpython-312.pyc
View File

Binary file not shown.

13
ddms_compliance_suite/input_parser/parser.py
View File

@ -4,6 +4,7 @@ from typing import List, Dict, Any, Optional, Union, Tuple
import requests
from urllib.parse import urljoin
import copy
import urllib3
logger = logging.getLogger(__name__)
@ -535,8 +536,14 @@ class InputParser:
self.logger.error(f"An unexpected error occurred while parsing Swagger spec {file_path}: {e}", exc_info=True)
return None
def parse_dms_spec(self, domain_mapping_path: str, base_url: str, headers: Optional[Dict[str, str]] = None) -> Optional[ParsedDMSSpec]:
def parse_dms_spec(self, domain_mapping_path: str, base_url: str, headers: Optional[Dict[str, str]] = None, ignore_ssl: bool = False) -> Optional[ParsedDMSSpec]:
self.logger.info(f"Starting DMS spec parsing. Base URL: {base_url}, Domain Map: {domain_mapping_path}")
if ignore_ssl:
self.logger.warning("SSL certificate verification is disabled for DMS API calls. This is not recommended for production use.")
# 禁用SSL警告
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
headers = headers or {}
try:
@ -549,7 +556,7 @@ class InputParser:
list_url = urljoin(base_url, "/api/schema/manage/schema")
self.logger.info(f"Fetching API list from: {list_url}")
try:
response = requests.get(list_url, headers=headers)
response = requests.get(list_url, headers=headers, verify=not ignore_ssl)
response.raise_for_status()
api_list_data = response.json()
@ -587,7 +594,7 @@ class InputParser:
self.logger.info(f"Fetching model for '{name}' from: {model_url}")
try:
response = requests.get(model_url, headers=headers)
response = requests.get(model_url, headers=headers, verify=not ignore_ssl)
response.raise_for_status()
model_schema_response = response.json()
except requests.exceptions.RequestException as e:

5
ddms_compliance_suite/test_orchestrator.py
View File

@ -2645,7 +2645,8 @@ class APITestOrchestrator:
def run_tests_from_dms(self, domain_mapping_path: str,
categories: Optional[List[str]] = None,
custom_test_cases_dir: Optional[str] = None
custom_test_cases_dir: Optional[str] = None,
ignore_ssl: bool = False
) -> Tuple[TestSummary, Optional[ParsedAPISpec]]:
"""
通过动态DMS服务发现来执行测试。
@ -2654,7 +2655,7 @@ class APITestOrchestrator:
parser = InputParser()
self.logger.info("从DMS动态服务启动测试...")
parsed_spec = parser.parse_dms_spec(domain_mapping_path, base_url=self.base_url)
parsed_spec = parser.parse_dms_spec(domain_mapping_path, base_url=self.base_url, ignore_ssl=ignore_ssl)
if not parsed_spec:
self.logger.error("无法从DMS服务解析API，测试终止。")

218
docs/SSL_Certificate_Handling.md Normal file
View File

@ -0,0 +1,218 @@
# SSL证书处理指南
## 🔒 问题描述
在连接到DMS服务时，可能会遇到SSL证书验证失败的错误：
```
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1010)'))
```
这通常发生在以下情况：
- 服务器使用自签名证书
- 证书链不完整
- 本地缺少根证书
- 开发/测试环境的证书配置问题
## 🛠️ 解决方案
### 方案1：忽略SSL证书验证（推荐用于开发/测试）
使用 `--ignore-ssl` 参数来忽略SSL证书验证：
```bash
python run_api_tests.py \
--dms ./assets/doc/dms/domain.json \
--base-url https://www.dev.ideas.cnpc \
--ignore-ssl \
--strictness-level CRITICAL
```
#### 特点：
- ✅ 快速解决SSL证书问题
- ✅ 适用于开发和测试环境
- ⚠️ 会显示安全警告
- ❌ 不推荐在生产环境使用
### 方案2：配置正确的SSL证书（推荐用于生产）
#### 2.1 安装根证书
```bash
# Ubuntu/Debian
sudo apt-get update
sudo apt-get install ca-certificates
# CentOS/RHEL
sudo yum update ca-certificates
# macOS
# 通过钥匙串访问添加证书
```
#### 2.2 添加自定义证书
如果服务器使用自签名证书，可以将证书添加到系统信任列表：
```bash
# 下载证书
openssl s_client -showcerts -connect www.dev.ideas.cnpc:443 </dev/null 2>/dev/null | openssl x509 -outform PEM > server.crt
# 添加到系统证书库（Ubuntu/Debian）
sudo cp server.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
# 添加到系统证书库（CentOS/RHEL）
sudo cp server.crt /etc/pki/ca-trust/source/anchors/
sudo update-ca-trust
```
#### 2.3 使用环境变量
```bash
# 设置证书文件路径
export REQUESTS_CA_BUNDLE=/path/to/certificate.pem
export SSL_CERT_FILE=/path/to/certificate.pem
# 运行测试
python run_api_tests.py --dms ./assets/doc/dms/domain.json
```
## 🧪 测试SSL配置
### 使用测试脚本
```bash
# 运行SSL测试脚本
python test_ssl_ignore.py
```
这个脚本会：
1. 测试使用 `--ignore-ssl` 参数的情况
2. 测试不使用 `--ignore-ssl` 参数的情况
3. 验证SSL忽略功能是否正常工作
### 手动测试
```bash
# 测试SSL连接
curl -v https://www.dev.ideas.cnpc/api/schema/manage/schema
# 忽略SSL测试
curl -k -v https://www.dev.ideas.cnpc/api/schema/manage/schema
```
## 📋 使用建议
### 开发环境
```bash
# 推荐使用SSL忽略
python run_api_tests.py \
--dms ./assets/doc/dms/domain.json \
--base-url https://www.dev.ideas.cnpc \
--ignore-ssl \
--strictness-level CRITICAL
```
### 测试环境
```bash
# 可以使用SSL忽略，但建议配置证书
python run_api_tests.py \
--dms ./assets/doc/dms/domain.json \
--base-url https://test.ideas.cnpc \
--ignore-ssl \
--strictness-level HIGH
```
### 生产环境
```bash
# 必须配置正确的SSL证书，不使用--ignore-ssl
python run_api_tests.py \
--dms ./assets/doc/dms/domain.json \
--base-url https://prod.ideas.cnpc \
--strictness-level CRITICAL
```
## 🔧 技术实现
### 代码修改
1. **添加命令行参数**：
```python
parser.add_argument('--ignore-ssl', action='store_true',
help='忽略SSL证书验证（不推荐在生产环境使用）')
```
2. **修改requests调用**：
```python
response = requests.get(url, verify=not ignore_ssl)
```
3. **禁用SSL警告**：
```python
if ignore_ssl:
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
```
### 影响的文件
- `run_api_tests.py` - 添加命令行参数
- `ddms_compliance_suite/input_parser/parser.py` - 修改SSL验证逻辑
- `ddms_compliance_suite/test_orchestrator.py` - 传递SSL参数
## ⚠️ 安全注意事项
### 风险
- **中间人攻击**：忽略SSL验证可能导致数据被截获
- **数据泄露**：敏感信息可能被恶意服务器获取
- **身份伪造**：无法验证服务器身份的真实性
### 最佳实践
1. **仅在开发/测试环境使用** `--ignore-ssl`
2. **生产环境必须配置正确的SSL证书**
3. **定期更新证书和根证书库**
4. **监控SSL证书过期时间**
5. **使用HTTPS代替HTTP**
## 🔍 故障排除
### 常见错误
1. **证书过期**
```
certificate verify failed: certificate has expired
```
解决：更新服务器证书
2. **主机名不匹配**
```
certificate verify failed: Hostname mismatch
```
解决：使用正确的主机名或配置SAN
3. **自签名证书**
```
certificate verify failed: self signed certificate
```
解决：添加证书到信任列表或使用 `--ignore-ssl`
### 调试命令
```bash
# 检查证书信息
openssl s_client -showcerts -connect www.dev.ideas.cnpc:443
# 验证证书链
openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt server.crt
# 测试连接
curl -v --cacert server.crt https://www.dev.ideas.cnpc
```
## 📚 相关资源
- [Python requests SSL文档](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification)
- [OpenSSL证书管理](https://www.openssl.org/docs/man1.1.1/man1/openssl-x509.html)
- [urllib3安全警告](https://urllib3.readthedocs.io/en/stable/advanced-usage.html#ssl-warnings)
通过合理配置SSL证书处理，可以确保API测试工具在各种环境中正常工作，同时保持适当的安全性。

11
run_api_tests.py
View File

@ -70,10 +70,12 @@ def parse_args():
filter_group.add_argument('--tags', help='Swagger标签，逗号分隔')
filter_group.add_argument('--list-categories', action='store_true', help='列出YAPI分类')
filter_group.add_argument('--list-tags', action='store_true', help='列出Swagger标签')
filter_group.add_argument('--strictness-level',
choices=['CRITICAL', 'HIGH', 'MEDIUM', 'LOW'],
default='CRITICAL',
filter_group.add_argument('--strictness-level',
choices=['CRITICAL', 'HIGH', 'MEDIUM', 'LOW'],
default='CRITICAL',
help='设置测试的严格等级。只有严重性等于或高于此级别的失败用例才会导致API端点被标记为失败。')
filter_group.add_argument('--ignore-ssl', action='store_true',
help='忽略SSL证书验证（不推荐在生产环境使用）')
# 新增：自定义测试用例参数组
custom_tc_group = parser.add_argument_group('自定义测试用例选项')
@ -829,7 +831,8 @@ def main():
test_summary, parsed_spec_for_scenarios = orchestrator.run_tests_from_dms(
domain_mapping_path=args.dms,
categories=categories,
custom_test_cases_dir=args.custom_test_cases_dir
custom_test_cases_dir=args.custom_test_cases_dir,
ignore_ssl=args.ignore_ssl
)
if not parsed_spec_for_scenarios: # 检查解析是否成功
logger.error(f"从DMS服务 '{args.dms}' 解析失败 (由编排器报告)。程序将退出。")

158
test_connection.py Executable file
View File

@ -0,0 +1,158 @@
#!/usr/bin/env python3
"""
测试DMS服务连接
"""
import requests
import urllib3
import json
import sys
def test_connection():
"""测试连接到DMS服务"""
base_url = "https://www.dev.ideas.cnpc"
api_url = f"{base_url}/api/schema/manage/schema"
print("🔧 测试DMS服务连接")
print("=" * 60)
print(f"目标URL: {api_url}")
print()
# 禁用SSL警告
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
# 测试1: 忽略SSL证书
print("📡 测试1: 忽略SSL证书验证")
try:
response = requests.get(api_url, verify=False, timeout=30)
print(f"✅ 连接成功!")
print(f"状态码: {response.status_code}")
print(f"响应头: {dict(response.headers)}")
if response.status_code == 200:
try:
data = response.json()
print(f"响应数据类型: {type(data)}")
if isinstance(data, dict):
print(f"响应键: {list(data.keys())}")
if 'code' in data:
print(f"业务代码: {data.get('code')}")
if 'message' in data:
print(f"消息: {data.get('message')}")
print("✅ JSON解析成功")
except json.JSONDecodeError as e:
print(f"⚠️ JSON解析失败: {e}")
print(f"响应内容前500字符: {response.text[:500]}")
else:
print(f"⚠️ HTTP状态码异常: {response.status_code}")
print(f"响应内容: {response.text[:500]}")
except requests.exceptions.SSLError as e:
print(f"❌ SSL错误: {e}")
return False
except requests.exceptions.ConnectionError as e:
print(f"❌ 连接错误: {e}")
return False
except requests.exceptions.Timeout as e:
print(f"❌ 超时错误: {e}")
return False
except Exception as e:
print(f"❌ 其他错误: {e}")
return False
print()
# 测试2: 启用SSL证书验证
print("📡 测试2: 启用SSL证书验证")
try:
response = requests.get(api_url, verify=True, timeout=30)
print(f"✅ SSL验证通过!")
print(f"状态码: {response.status_code}")
except requests.exceptions.SSLError as e:
print(f"❌ SSL验证失败（预期）: {e}")
print("这证明SSL忽略功能是必要的")
except Exception as e:
print(f"❌ 其他错误: {e}")
print()
# 测试3: 测试基础连接
print("📡 测试3: 测试基础HTTP连接")
try:
# 尝试连接到根路径
root_url = base_url
response = requests.get(root_url, verify=False, timeout=10)
print(f"根路径连接: {response.status_code}")
except Exception as e:
print(f"根路径连接失败: {e}")
return True
def test_domain_mapping():
"""测试域映射文件"""
print("📁 测试域映射文件")
print("=" * 60)
domain_file = "./assets/doc/dms/domain.json"
try:
with open(domain_file, 'r', encoding='utf-8') as f:
domain_data = json.load(f)
print(f"✅ 域映射文件读取成功")
print(f"文件路径: {domain_file}")
print(f"域映射数据: {domain_data}")
return True
except FileNotFoundError:
print(f"❌ 域映射文件不存在: {domain_file}")
return False
except json.JSONDecodeError as e:
print(f"❌ 域映射文件JSON格式错误: {e}")
return False
except Exception as e:
print(f"❌ 读取域映射文件出错: {e}")
return False
def main():
"""主函数"""
print("🧪 DMS服务连接测试")
print("=" * 80)
success = True
# 测试域映射文件
if not test_domain_mapping():
success = False
print()
# 测试连接
if not test_connection():
success = False
print("=" * 80)
if success:
print("🎉 连接测试完成")
print("\n💡 建议:")
print("- 如果SSL验证失败但忽略SSL成功，使用 --ignore-ssl 参数")
print("- 如果连接完全失败，检查网络和防火墙设置")
print("- 如果JSON解析失败，检查API端点是否正确")
else:
print("❌ 连接测试失败")
print("\n🔧 故障排除:")
print("1. 检查网络连接")
print("2. 检查防火墙设置")
print("3. 确认服务器地址正确")
print("4. 检查域映射文件是否存在")
return success
if __name__ == "__main__":
if main():
sys.exit(0)
else:
sys.exit(1)

181
test_ssl_ignore.py Executable file
View File

@ -0,0 +1,181 @@
#!/usr/bin/env python3
"""
测试SSL证书忽略功能
"""
import sys
import subprocess
import logging
# 设置日志
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
def test_ssl_ignore():
"""测试SSL忽略功能"""
print("🔧 测试SSL证书忽略功能")
print("=" * 60)
# 测试命令
test_command = [
"python", "run_api_tests.py",
"--dms", "./assets/doc/dms/domain.json",
"--base-url", "https://www.dev.ideas.cnpc",
"--ignore-ssl",
"--strictness-level", "CRITICAL",
"--output", "./test_reports/ssl_test",
"--format", "json"
]
print("🚀 执行测试命令:")
print(" ".join(test_command))
print()
try:
# 执行测试
result = subprocess.run(
test_command,
capture_output=True,
text=True,
timeout=300 # 5分钟超时
)
print("📊 测试结果:")
print(f"返回码: {result.returncode}")
print()
if result.stdout:
print("📝 标准输出:")
print(result.stdout)
print()
if result.stderr:
print("⚠️ 错误输出:")
print(result.stderr)
print()
# 分析结果
if result.returncode == 0:
print("✅ SSL忽略功能测试成功！")
return True
else:
print("❌ SSL忽略功能测试失败")
# 检查是否还有SSL错误
if "SSL" in result.stderr or "certificate" in result.stderr.lower():
print("🔍 仍然存在SSL相关错误，可能需要进一步调试")
else:
print("🔍 SSL错误已解决，但可能存在其他问题")
return False
except subprocess.TimeoutExpired:
print("⏰ 测试超时（5分钟）")
return False
except Exception as e:
print(f"❌ 测试执行出错: {e}")
return False
def test_without_ssl_ignore():
"""测试不使用SSL忽略的情况（应该失败）"""
print("\n🔧 测试不忽略SSL证书（预期失败）")
print("=" * 60)
# 测试命令（不包含--ignore-ssl）
test_command = [
"python", "run_api_tests.py",
"--dms", "./assets/doc/dms/domain.json",
"--base-url", "https://www.dev.ideas.cnpc",
"--strictness-level", "CRITICAL",
"--output", "./test_reports/ssl_test_no_ignore",
"--format", "json"
]
print("🚀 执行测试命令（不忽略SSL）:")
print(" ".join(test_command))
print()
try:
# 执行测试
result = subprocess.run(
test_command,
capture_output=True,
text=True,
timeout=60 # 1分钟超时，应该很快失败
)
print("📊 测试结果:")
print(f"返回码: {result.returncode}")
print()
if result.stderr:
print("⚠️ 错误输出:")
print(result.stderr[:500] + "..." if len(result.stderr) > 500 else result.stderr)
print()
# 分析结果
if result.returncode != 0 and ("SSL" in result.stderr or "certificate" in result.stderr.lower()):
print("✅ 预期的SSL错误出现，证明SSL验证正常工作")
return True
else:
print("⚠️ 未出现预期的SSL错误，可能配置有问题")
return False
except subprocess.TimeoutExpired:
print("⏰ 测试超时，可能SSL验证导致连接挂起")
return True # 这也算是预期行为
except Exception as e:
print(f"❌ 测试执行出错: {e}")
return False
def main():
"""主函数"""
print("🧪 DMS合规性测试工具 - SSL证书忽略功能测试")
print("=" * 80)
# 检查必要文件
import os
if not os.path.exists("./assets/doc/dms/domain.json"):
print("❌ 找不到DMS域映射文件: ./assets/doc/dms/domain.json")
print("请确保文件存在后重试")
sys.exit(1)
if not os.path.exists("run_api_tests.py"):
print("❌ 找不到主测试脚本: run_api_tests.py")
sys.exit(1)
# 创建测试报告目录
os.makedirs("./test_reports", exist_ok=True)
success_count = 0
total_tests = 2
# 测试1: 使用SSL忽略
if test_ssl_ignore():
success_count += 1
# 测试2: 不使用SSL忽略（预期失败）
if test_without_ssl_ignore():
success_count += 1
# 总结
print("\n" + "=" * 80)
print("📋 测试总结")
print("=" * 80)
print(f"通过测试: {success_count}/{total_tests}")
if success_count == total_tests:
print("🎉 所有测试通过！SSL忽略功能工作正常")
print("\n💡 使用建议:")
print("- 在开发和测试环境中使用 --ignore-ssl 参数")
print("- 在生产环境中不要使用此参数，确保SSL证书验证")
print("- 如果需要在生产环境中使用，请配置正确的SSL证书")
sys.exit(0)
else:
print("❌ 部分测试失败，请检查配置")
sys.exit(1)
if __name__ == "__main__":
main()