runningInConsole()) { $this->isAuth = false; } parent::__construct(); $this->model = FileModel::class; $this->selectField = [ 'id', 'user_id', 'oss_config_id', 'folder_id', 'name', 'url', 'path', 'ext', 'type', 'size', 'width', 'height', 'hash', 'ref_count', 'last_scan_at', 'source', 'created_at', 'updated_at', ]; $this->queryField = [ 'folder_id' => '=', 'type' => '=', 'ext' => '=', 'oss_config_id' => '=', 'source' => '=', ]; $this->media = MediaUrlService::getInstance(); } /** * 素材列表 * * keyword 要同时命中 name / path / url,而基类的 queryField 只会按列 AND, * 拼不出这个 OR 组,所以这里自己组查询,返回结构与 getPageList 保持一致。 * * @throws ContainerExceptionInterface * @throws NotFoundExceptionInterface */ public function list(): array { $this->getWhere(); $keyword = trim((string) request()->get('keyword', '')); $unused = (int) request()->get('unused', 0); $searchTime = request()->get('search_time'); $query = FileModel::where($this->where) ->when($keyword !== '', function ($q) use ($keyword) { $q->where(function ($sub) use ($keyword) { $sub->where('name', 'like', '%' . $keyword . '%') ->orWhere('path', 'like', '%' . $keyword . '%') ->orWhere('url', 'like', '%' . $keyword . '%'); }); }) ->when($unused === 1, fn ($q) => $q->where('ref_count', 0)) ->when(!empty($searchTime), function ($q) use ($searchTime) { $this->getWhereBetween($searchTime); $q->whereBetween($this->whereBetween[0], $this->whereBetween[1]); }); return $this->toPage($query->select($this->selectField)->orderByDesc('id')); } /** * 详情,带上文件夹名,免得前端为了显示一个名字再请求一次目录树 * * @throws Exception */ public function detail($id): mixed { $info = $this->getDetail($id); $info->url = $this->media->toPublic($info->url); $info->folder_name = (string) (FileFolderModel::where('id', (int) $info->folder_id)->value('name') ?? ''); return $info; } /** * 只允许改素材名与所属文件夹 * * path / hash / size 是同步结果,手工改会让引用扫描直接失准: * path 一旦被改歪,原本在用的素材就匹配不上任何引用,转头出现在可回收列表里。 * * @throws Exception */ public function update($id, $params): mixed { $data = []; if (array_key_exists('name', $params)) { $data['name'] = mb_substr(trim((string) $params['name']), 0, 255); } if (array_key_exists('folder_id', $params)) { $data['folder_id'] = $this->assertFolder((int) $params['folder_id']); } if (empty($data)) { $this->utils->errorThrow('没有可更新的内容'); } return $this->save($id, $data); } /** * 只软删素材记录,不动远端对象 * * 远端删除必须走 reclaim 的二次校验;这里如果顺手删了远端, * 误删的文件就再也找不回来了,而软删记录随时可以恢复。 * * @throws Exception */ public function delete($ids): mixed { return $this->del(is_array($ids) ? $ids : [$ids]); } /** * 素材概览:总量、占用空间与可回收部分 */ public function stat(): array { $base = static fn () => FileModel::where('deleted_at', 0); $types = $base()->selectRaw('type, COUNT(*) as count') ->groupBy('type') ->orderBy('type') ->get() ->map(static fn ($row) => ['type' => (int) $row->type, 'count' => (int) $row->count]) ->all(); return [ 'total' => $base()->count(), 'total_size' => (int) $base()->sum('size'), 'unused' => $base()->where('ref_count', 0)->count(), 'unused_size' => (int) $base()->where('ref_count', 0)->sum('size'), 'types' => $types, ]; } /** * 从 OSS 增量拉对象进素材表 * * 一次只处理一页:返回的 next_marker 非空就带着它再调一次,前端点几下即可拉完。 * 幂等靠三级匹配 —— 先按对象键,再按完整地址(换过域名的老记录), * 最后才按哈希兜住「地址早就变了、内容没变」的历史上传记录。 * 反复调用只会更新 size / hash,不会重复插入。 * * @param array $params oss_config_id / prefix / marker / limit * @throws Exception */ public function syncFromOss(array $params): array { $configId = (int) ($params['oss_config_id'] ?? 0); if ($configId <= 0) { $this->utils->errorThrow('请选择要同步的存储配置'); } $limit = (int) ($params['limit'] ?? self::SYNC_DEFAULT_LIMIT); $limit = $limit > 0 ? min($limit, self::SYNC_MAX_LIMIT) : self::SYNC_DEFAULT_LIMIT; $page = $this->driverOf($configId)->listObjects( (string) ($params['prefix'] ?? ''), (string) ($params['marker'] ?? ''), $limit ); $items = array_values(array_filter( (array) ($page['items'] ?? []), static fn ($item) => !empty($item['key']) && !str_ends_with((string) $item['key'], '/') )); $result = [ 'inserted' => 0, 'updated' => 0, 'next_marker' => (string) ($page['next_marker'] ?? ''), 'finished' => (bool) ($page['finished'] ?? true), ]; if (empty($items)) { return $result; } [$byPath, $byUrl, $byHash] = $this->existingIndexOf($items); $now = time(); $pending = []; $seen = []; foreach ($items as $item) { $key = (string) $item['key']; if (isset($seen[$key])) { continue; } $seen[$key] = true; $url = (string) ($item['url'] ?? ''); $hash = (string) ($item['hash'] ?? ''); $ext = strtolower((string) pathinfo($key, PATHINFO_EXTENSION)); $row = $byPath[$key] ?? $byUrl[$url] ?? ($hash !== '' ? ($byHash[$hash] ?? null) : null); if ($row !== null) { FileModel::where('id', $row->id)->update( $this->backfillOf($row, $key, $url, $hash, $ext, (int) ($item['size'] ?? 0), $configId, $now) ); $result['updated']++; continue; } $pending[] = [ 'user_id' => $this->userId, 'oss_config_id' => $configId, 'folder_id' => 0, 'name' => basename($key), 'url' => $url, 'path' => $key, 'ext' => $ext, 'type' => FileModel::typeOfExt($ext), 'size' => (int) ($item['size'] ?? 0), 'width' => 0, 'height' => 0, 'hash' => $hash, 'ref_count' => 0, // 新补录的素材必须是 0:非 0 会被 reclaim 当成「扫过且没人用」直接删掉 'last_scan_at' => 0, 'source' => FileModel::SOURCE_OSS, // 用对象的实际修改时间当上传时间,否则批量补录出来的素材 // 创建时间全挤在同一秒,「N 天前的无引用文件」这个筛选就没意义了 'created_at' => (int) ($item['last_modified'] ?? 0) ?: $now, 'updated_at' => 0, 'deleted_at' => 0, ]; } if (!empty($pending)) { FileModel::insert($pending); $result['inserted'] = count($pending); } return $result; } /** * 全库引用扫描,回填 ref_count 与 last_scan_at * * 匹配策略:先按完整对象键,再退回文件名兜底。上传时的文件名是随机串, * 现实中不会撞车;真撞车(同名多条)就放弃文件名兜底,宁可少算一次引用, * 也不能把引用记到错的素材头上 —— 记错的那一头会被判成可回收。 * * 软删的业务行也照样算引用:那些记录随时可能被恢复, * 把它们的封面提前删掉等于让恢复出来的数据全是裂图。 */ public function scanReferences(array $params = []): array { [$byPath, $byName] = $this->buildMaterialIndex(); $counts = []; $scanned = 0; foreach ((array) config('media_refs', []) as $ref) { $conn = (string) ($ref['connection'] ?? 'mysql'); $table = (string) ($ref['table'] ?? ''); $field = (string) ($ref['field'] ?? ''); $kind = (string) ($ref['kind'] ?? 'single'); if ($table === '' || $field === '') { continue; } try { $schema = Schema::connection($conn); if (!$schema->hasTable($table) || !$schema->hasColumn($table, $field)) { // 登记了但库里还没这列(比如 catalogue.video 尚未上线):跳过而不是报错, // 否则一个规划中的字段就能让整次扫描前功尽弃 continue; } } catch (Throwable $e) { continue; } DB::connection($conn)->table($table) ->select(['id', $field]) ->orderBy('id') ->chunk(self::CHUNK_SIZE, function ($rows) use ($field, $kind, $byPath, $byName, &$counts, &$scanned) { foreach ($rows as $row) { $raw = $row->{$field} ?? null; if ($raw === null || trim((string) $raw) === '') { continue; } foreach ($this->extractUrls((string) $raw, $kind) as $url) { $key = $this->normalizeRefKey($url); if ($key === '') { continue; } $scanned++; $id = $byPath[$key] ?? ($byName[basename($key)] ?? null); if ($id !== null) { $counts[$id] = ($counts[$id] ?? 0) + 1; } } } }); } $now = time(); // 先整表归零并盖上扫描时间戳,再把有引用的批量改回去: // 逐条 update 在几万条素材上是几万次往返 FileModel::where('deleted_at', 0)->update(['ref_count' => 0, 'last_scan_at' => $now]); $grouped = []; foreach ($counts as $id => $count) { $grouped[(int) $count][] = (int) $id; } foreach ($grouped as $count => $ids) { foreach (array_chunk($ids, self::CHUNK_SIZE) as $slice) { FileModel::whereIn('id', $slice)->update(['ref_count' => $count]); } } return [ 'scanned' => $scanned, 'referenced' => count($counts), 'unused' => FileModel::where('deleted_at', 0)->where('ref_count', 0)->count(), ]; } /** * 无引用素材清单 * * 除了 ref_count = 0 还要卡「上传超过 N 天」:用户上传完图片、表单还没提交时, * 这张图确实没人引用,但它马上就要被用上,不能出现在回收列表里。 */ public function unusedList(array $params = []): array { $days = (int) ($params['days'] ?? 30); $days = $days > 0 ? $days : 30; $query = FileModel::where('deleted_at', 0) ->where('ref_count', 0) ->where('created_at', '<', time() - $days * 86400) ->select($this->selectField) // 先给大文件,回收一页就能腾出可观的空间 ->orderByDesc('size') ->orderByDesc('id'); return $this->toPage($query, (int) ($params['pageSize'] ?? 20)); } /** * 回收:删远端对象,成功后软删记录 * * 两道闸门都不能省 —— ref_count = 0 说明扫描时没人用,last_scan_at > 0 说明真的扫过。 * 单条失败不中断整批:一批几百个对象,前面已经删掉的必须留下软删记录, * 否则库里还挂着记录、远端对象已经没了,素材库里全是点开 404 的幽灵。 * * @param array $ids 素材 ID * @throws Exception */ public function reclaim(array $ids): array { $ids = array_values(array_unique(array_filter(array_map('intval', $ids)))); if (empty($ids)) { $this->utils->errorThrow('请选择要回收的素材'); } $rows = FileModel::whereIn('id', $ids)->where('deleted_at', 0)->get(); $deleted = 0; $failed = []; $drivers = []; foreach ($rows as $row) { $id = (int) $row->id; if ((int) $row->last_scan_at <= 0) { $failed[] = ['id' => $id, 'reason' => '尚未扫描过引用,请先执行引用扫描']; continue; } if ((int) $row->ref_count !== 0) { $failed[] = ['id' => $id, 'reason' => '仍被引用 ' . (int) $row->ref_count . ' 处']; continue; } $key = trim((string) $row->path) !== '' ? (string) $row->path : $this->normalizeRefKey((string) $row->url); if ($key === '') { $failed[] = ['id' => $id, 'reason' => '没有可定位的对象键,请先同步一次']; continue; } try { $configId = (int) $row->oss_config_id; if (!isset($drivers[$configId])) { $drivers[$configId] = $this->driverOf($configId); } if (!$drivers[$configId]->deleteObject($key)) { $failed[] = ['id' => $id, 'reason' => '远端删除失败']; continue; } } catch (Throwable $e) { $failed[] = ['id' => $id, 'reason' => $e->getMessage()]; continue; } $now = time(); FileModel::where('id', $id)->update(['deleted_at' => $now, 'updated_at' => $now]); $deleted++; } $found = $rows->pluck('id')->map(static fn ($value) => (int) $value)->all(); foreach (array_diff($ids, $found) as $missing) { $failed[] = ['id' => (int) $missing, 'reason' => '素材不存在或已删除']; } return ['deleted' => $deleted, 'failed' => array_values($failed)]; } /** * 批量移动到文件夹(folder_id = 0 表示移回根目录) * * @throws Exception */ public function moveToFolder(array $ids, int $folderId): array { $ids = array_values(array_unique(array_filter(array_map('intval', $ids)))); if (empty($ids)) { $this->utils->errorThrow('请选择要移动的素材'); } $folderId = $this->assertFolder($folderId); $moved = FileModel::whereIn('id', $ids)->where('deleted_at', 0)->update([ 'folder_id' => $folderId, 'updated_at' => time(), ]); return ['moved' => (int) $moved, 'folder_id' => $folderId]; } /** * 分页结果统一成前端约定的结构 */ private function toPage($query, int $pageSize = 0): array { $pageSize = $pageSize > 0 ? $pageSize : (int) request()->get('pageSize', 20); $result = $query->paginate($pageSize > 0 ? $pageSize : 20)->toArray(); foreach ($result['data'] as &$item) { $item['url'] = $this->media->toPublic($item['url'] ?? ''); } unset($item); return [ 'page' => $result['current_page'], 'size' => $result['per_page'], 'page_count' => $result['last_page'], 'total' => $result['total'], 'items' => $result['data'], ]; } /** * 一次把本批可能命中的已有记录捞出来,避免逐条 select 打 N 次库 * * byHash 只收 path still 为空的行:按内容哈希去认亲很容易把「两个键、同一份内容」 * 的对象合成一条,只用来给还没记过对象键的历史上传记录补档。 * * @param array $items * @return array{0: array, 1: array, 2: array} */ private function existingIndexOf(array $items): array { $paths = array_values(array_unique(array_column($items, 'key'))); $urls = array_values(array_unique(array_filter(array_column($items, 'url')))); $hashes = array_values(array_unique(array_filter(array_column($items, 'hash')))); $rows = FileModel::where('deleted_at', 0) ->where(function ($q) use ($paths, $urls, $hashes) { $q->whereIn('path', $paths); if (!empty($urls)) { $q->orWhereIn('url', $urls); } if (!empty($hashes)) { $q->orWhere(function ($sub) use ($hashes) { $sub->where('path', '')->whereIn('hash', $hashes); }); } }) ->get(['id', 'name', 'url', 'path', 'ext', 'type', 'hash', 'oss_config_id']); $byPath = []; $byUrl = []; $byHash = []; foreach ($rows as $row) { $path = (string) $row->path; $url = (string) $row->url; $hash = (string) $row->hash; if ($path !== '') { $byPath[$path] = $row; } if ($url !== '' && !isset($byUrl[$url])) { $byUrl[$url] = $row; } if ($path === '' && $hash !== '' && !isset($byHash[$hash])) { $byHash[$hash] = $row; } } return [$byPath, $byUrl, $byHash]; } /** * 已有记录的更新字段 * * size / hash 每次都覆盖(对象可能被同名替换过),其余列只在原值为空时补, * 免得把用户在素材库里改过的名字、归过的文件夹又冲回默认值。 */ private function backfillOf( mixed $row, string $key, string $url, string $hash, string $ext, int $size, int $configId, int $now ): array { $update = ['size' => $size, 'updated_at' => $now]; if ($hash !== '') { $update['hash'] = $hash; } if (trim((string) $row->path) === '') { $update['path'] = $key; } if (trim((string) $row->url) === '' && $url !== '') { $update['url'] = $url; } if (trim((string) $row->ext) === '' && $ext !== '') { $update['ext'] = $ext; $update['type'] = FileModel::typeOfExt($ext); } if (trim((string) $row->name) === '') { $update['name'] = basename($key); } if ((int) $row->oss_config_id === 0) { $update['oss_config_id'] = $configId; } return $update; } /** * 素材索引:对象键 → id,文件名 → id * * 文件名索引里同名多条时置 null(放弃兜底),见 scanReferences 的说明。 * * @return array{0: array, 1: array} */ private function buildMaterialIndex(): array { $byPath = []; $byName = []; FileModel::where('deleted_at', 0) ->select(['id', 'path', 'url']) ->orderBy('id') ->chunk(self::CHUNK_SIZE, function ($rows) use (&$byPath, &$byName) { foreach ($rows as $row) { $id = (int) $row->id; // path 与 url 都进索引:老记录只有 url,新记录两者都有 foreach ([(string) $row->path, (string) $row->url] as $candidate) { $key = $this->normalizeRefKey($candidate); if ($key === '') { continue; } $byPath[$key] = $id; $name = basename($key); if ($name === '') { continue; } $byName[$name] = array_key_exists($name, $byName) && $byName[$name] !== $id ? null : $id; } } }); return [$byPath, $byName]; } /** * 把引用地址收敛成能与素材 path 比对的对象键 * * 同一个文件在库里可能是绝对地址、/storage 相对地址、带 ?imageView2 处理参数 * 三种写法,不先归一化就只能匹配上碰巧写法一致的那批。 */ private function normalizeRefKey(string $value): string { $value = $this->media->stripProcessParams($value); if ($value === '') { return ''; } $path = (string) (parse_url($value, PHP_URL_PATH) ?: $value); $path = ltrim((string) preg_replace('#/{2,}#', '/', rawurldecode($path)), '/'); // 本地存储出库地址带 /storage 前缀,对象键里没有 if (str_starts_with($path, 'storage/')) { $path = substr($path, 8); } return $path; } /** * 按登记的 kind 从字段值里取出地址 * * @return array */ private function extractUrls(string $value, string $kind): array { $value = trim($value); if ($value === '') { return []; } return match ($kind) { 'multi' => array_values(array_filter( array_map('trim', explode(',', $value)), static fn ($item) => $item !== '' )), 'rich' => $this->extractFromRichText($value), default => [$value], }; } /** * 富文本 / Markdown 里的地址 * * 三种写法都要抽:HTML 属性(src/href/poster)、内联样式的 url(…)、 * Markdown 的 ![](…)。只抽 src 会漏掉正文里手写的 Markdown 图片。 * * @return array */ private function extractFromRichText(string $content): array { $found = []; $patterns = [ '#(?:src|href|poster|data-src)\s*=\s*[\'"]([^\'"]+)[\'"]#i', '#url\(\s*[\'"]?([^\'")]+)[\'"]?\s*\)#i', '#!\[[^\]]*\]\(\s*([^\s)]+)#', ]; foreach ($patterns as $pattern) { if (preg_match_all($pattern, $content, $matches)) { foreach ($matches[1] as $hit) { $hit = trim((string) $hit); if ($hit !== '') { $found[] = $hit; } } } } return array_values(array_unique($found)); } /** * 取指定配置的存储驱动 * * oss_config_id 为 0 的是扩表之前的老记录,只能按当前启用配置去删; * 换过存储的站点这类记录得先同步一次把 config_id 补上,否则会删错 bucket。 * * @throws Exception */ private function driverOf(int $configId): OssStorageInterface { $runtime = OssRuntimeConfigService::getInstance(); $config = $configId > 0 ? $runtime->getConfigById($configId) : $runtime->getActiveConfig(); return OssStorageFactory::getInstance()->make($config); } /** * 0 表示根目录,其余必须是存在的文件夹 * * 不校验就会把素材移进一个查不到的 folder_id,那批文件在素材库里再也点不出来 * * @throws Exception */ private function assertFolder(int $folderId): int { if ($folderId <= 0) { return 0; } if (!FileFolderModel::where('id', $folderId)->where('deleted_at', 0)->exists()) { $this->utils->notFound('文件夹不存在'); } return $folderId; } }