Nestjs框架: 基于Mongodb的多租户功能集成和优化

关注

原创 2025-07-27 10:11:56 833 阅读 AI 写同款· GEO 优化 ›

Nestjs框架: 数据库多租户模式与动态模块初探本文探讨了如何通过动态模块和依赖注入实现多租户数据库系统的集成。文章首先概述了TypeORM、Prisma和Mongoose三种ORM框架的基本连接方式，重点分析了Prisma在多库场景下的动态配置需求。通过研究Prisma的文档，提出了通过程序化覆盖数据源URL的方案，将连接信息拆分为独立字段管理。随后深入剖析了NestJS动态模块的实现原理，通过分析TypeORM模块源码和官方文档示例，展示了如何通过forRootAsync方法实现配置的动态注入。最后，文章通过一个多租户场景案例，阐述了如何利用动态模块阅读详情

概述

基于前文，我们知道如何集成多租户的相关功能了, 现在我们继续集成Monodb的多租户形式
需要注意的是，MongoDB 在 NestJS 中的使用过程中存在一些“坑点”
如果按照默认方式集成，会发现连接数在不断增长，即使我们请求的是相同的数据库地址和租户信息
这说明每次接口请求都在创建新的数据库连接，而不是复用已有连接，这个行为与我们预期不符
数据库连接是有限资源，频繁创建连接会导致 IO 资源耗尽、性能下降等问题
下面，我们分别来看看如何解决此类问题

开始集成

1 ) 编写 docker-compose.multi.yaml 文件

services:
  mongo:
    image: mongo:8  # 使用最新的 MongoDB 镜像
    container_name: mongo_app
    restart: always
    ports:
      - "27018:27017"  # 将容器的 27017 端口映射到主机的 27017 端口
    environment:
      - MONGO_INITDB_ROOT_USERNAME=root  # 设置 MongoDB 的 root 用户名
      - MONGO_INITDB_ROOT_PASSWORD=123456_mongodb  # 设置 MongoDB 的 root 密码
      # 调整日志级别的例子，可选值如 "0"（致命错误）、"1"（警告+错误）、"2"（信息+警告+错误）...
      - MONGO_LOG_LEVEL=2
      - MONGO_SYSTEM_LOG_PATH=/data/logs/mongodb.log
    volumes:
      - ./docker-dbconfig/mongo/conf/mongod.conf:/etc/mongod.conf
      - ./docker-dbconfig/mongo/data/db:/data/db  # 将容器内的 /data/db 目录映射到本地的 ./data/db 目录，用于数据持久化
      - ./docker-dbconfig/mongo/logs:/data/logs
    networks:
      - light_network

  mongo-express:
    image: mongo-express:latest
    container_name: mongo_express_app
    restart: always
    environment:
      ME_CONFIG_MONGODB_ADMINUSERNAME: root
      ME_CONFIG_MONGODB_ADMINPASSWORD: 123456_mongodb
      ME_CONFIG_MONGODB_URL: mongodb://root:123456_mongodb@mongo:27017/
      ME_CONFIG_BASICAUTH: false
    ports:
      - 18081:8081
    networks:
      - light_network

  mongo2:
    image: mongo:8  # 使用最新的 MongoDB 镜像
    container_name: mongo_app2
    restart: always
    ports:
      - "27019:27017"  # 将容器的 27017 端口映射到主机的 27017 端口
    environment:
      - MONGO_INITDB_ROOT_USERNAME=root  # 设置 MongoDB 的 root 用户名
      - MONGO_INITDB_ROOT_PASSWORD=123456_mongodb  # 设置 MongoDB 的 root 密码
      # 调整日志级别的例子，可选值如 "0"（致命错误）、"1"（警告+错误）、"2"（信息+警告+错误）...
      - MONGO_LOG_LEVEL=2
      - MONGO_SYSTEM_LOG_PATH=/data/logs/mongodb.log
    volumes:
      - ./docker-dbconfig/mongo2/conf/mongod.conf:/etc/mongod.conf
      - ./docker-dbconfig/mongo2/data/db:/data/db  # 将容器内的 /data/db 目录映射到本地的 ./data/db 目录，用于数据持久化
      - ./docker-dbconfig/mongo2/logs:/data/logs
    networks:
      - light_network

  mongo-express2:
    image: mongo-express:latest
    container_name: mongo_express_app2
    restart: always
    environment:
      ME_CONFIG_MONGODB_ADMINUSERNAME: root
      ME_CONFIG_MONGODB_ADMINPASSWORD: 123456_mongodb
      ME_CONFIG_MONGODB_URL: mongodb://root:123456_mongodb@mongo2:27017/
      ME_CONFIG_BASICAUTH: false
    ports:
      - 18082:8081
    networks:
      - light_network

networks:
  light_network:
    external: true

启动服务，之后在 UI 管理界面给 2个数据库加入可识别的数据
下面测试的时候，会看到数据

2 ) 配置 .env

T1_MONGODB_URI="mongodb://root:123456_mongodb@localhost:27018/nestmongodb"
T2_MONGODB_URI="mongodb://root:123456_mongodb@localhost:27019/nestmongodb"

3 ) 编写 database/mongoose/mongoose.constant.ts

// 这个配置模拟调接口/读数据库获取的
export const tenantMap = new Map([
	['1', 'T1'],
	['2', 'T2']
]);
export const defaultTenant = tenantMap.values().next().value; // 第一个

4 ) 编写 database/mongoose/mongoose-config.service.ts

import { Inject } from '@nestjs/common';
import { REQUEST } from '@nestjs/core';
import {
	MongooseModuleOptions,
	MongooseOptionsFactory,
} from '@nestjs/mongoose';
import { Request } from 'express';
import { tenantMap, defaultTenant } from './mongoose.constant';
import { ConfigService } from '@nestjs/config';

export class MongooseConfigService implements MongooseOptionsFactory {
	constructor(
		@Inject(REQUEST) private request: Request,
		private configService: ConfigService,
	) {}
	
	createMongooseOptions():
		| MongooseModuleOptions
		| Promise<MongooseModuleOptions> {
			const headers = this.request.headers;
			const tenantId = headers['x-tenant-id'] as string;
			console.log('tenantId: ', tenantId)
			if (tenantId && !tenantMap.has(tenantId)) {
				throw new Error('invalid tenantId');
			}
			const t_prefix = !tenantId ? defaultTenant : tenantMap.get(tenantId);
			const uri = this.configService.get<string>(`${t_prefix}_MONGODB_URI`);
			return { uri } as MongooseModuleOptions;
		}
}

5 ) 编写 app.module.ts

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { MongooseModule } from '@nestjs/mongoose';
import { UserSchema } from './user/user.schema'
import { ConfigModule, ConfigService } from '@nestjs/config';
import { MongooseConfigService } from './database/mongoose/mongoose-config.service';

@Module({
  imports: [
    // 1. 下面这个后续可以封装一个新的模块，来匹配 .env 和 其他配置
    ConfigModule.forRoot({  // 配置环境变量模块
      envFilePath: '.env', // 指定环境变量文件路径
      isGlobal: true, // 全局可用
    }),

    MongooseModule.forRootAsync({
      useClass: MongooseConfigService,
    }),
    MongooseModule.forFeature([{ name: 'User', schema: UserSchema }]),
  ],
  controllers: [AppController],
  providers: []
})

export class AppModule {}

6 ) 编写 app.controller.ts

import { Controller, Get } from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { User } from './user/user.entity';
import { InjectModel } from '@nestjs/mongoose';
import { Model, Document as Doc } from 'mongoose';

@Controller()
export class AppController {
  constructor(
    @InjectModel('User') private userModel: Model<Doc>,
  ) {}

  @Get('/multi-mongoose')
  async getMongooseUsers(): Promise<any> {
    const rs = await this.userModel.find()
    return rs;
  }
}

测试

1 ) 测试1

请求

curl --request GET \
  --url http://localhost:3000/multi-mongoose \
  --header 'x-tenant-id: 1'

响应

[
  {
    "_id": "6874d4b0d10e36e350dd588d",
    "username": "mongo1",
    "password": "123456"
  },
  {
    "_id": "6874d4d9d10e36e350dd588f",
    "username": "lee",
    "password": "123456"
  }
]

2 ) 测试2

请求

curl --request GET \
  --url http://localhost:3000/multi-mongoose \
  --header 'x-tenant-id: 2'

响应

[
  {
    "_id": "6874d4b0d10e36e350dd588d",
    "username": "mongo2",
    "password": "123456"
  },
  {
    "_id": "6874d4d9d10e36e350dd588f",
    "username": "lee",
    "password": "123456"
  }
]

3 ）进入其中一个数据库，测试连接情况

docker exec -it mongo_app2 mongosh admin -u root # 输入密码
use nestmongodb;

# 其实上面一个 use 命令 可以不用
db.serverStatus().connections

输出如下:

{
  current: 6,
  available: 838839,
  totalCreated: 38,
  rejected: 0,
  active: 2,
  threaded: 6,
  exhaustIsMaster: Long('0'),
  exhaustHello: Long('0'),
  awaitingTopologyChanges: Long('1'),
  loadBalanced: Long('0')
}

目前 active 有2个，当不断请求同一个接口，在此执行上述命令，可发现这个数字是累加的
这明显是不对的，访问相同链接应该使用同一个 connection, 而不是创建新的通道
当租户多起来的时候，会带来严重的性能问题

4 ) 分析原因

在 @nestjs/mongoose 包的 mongoose-core.module.ts 中在使用 factory 方法的时候，会调用 createMongooseConnection

private static async createMongooseConnection(
   uri: string,
   mongooseOptions: ConnectOptions,
   factoryOptions: {
     lazyConnection?: boolean;
     onConnectionCreate?: MongooseModuleOptions['onConnectionCreate'];
   },
): Promise<Connection> {
   const connection = mongoose.createConnection(uri, mongooseOptions);

   if (factoryOptions?.lazyConnection) {
     return connection;
   }

   factoryOptions?.onConnectionCreate?.(connection);

   return connection.asPromise();
}

这个是每次都会创建的根本原因，这个包也没有提供类似 datasourceFactory 的功能
现在需要定制 mongoose 的 module 中的 forRootSync 的逻辑

定制 mongoose 的 forRootAsync 方法

现在需要对官方 @nestjs/mongoose 包中的核心模块的方法进行裁剪和优化
找到对应的文件贴到自己项目中进行修改

1 ）新建 src/database/mongoose/mongoose.module.ts

import { Module, DynamicModule } from '@nestjs/common';
import { 
	MongooseModuleAsyncOptions, 
	MongooseModuleOptions, 
	MongooseModule as NestMongooseModule
} from '@nestjs/mongoose';
import { MongooseCoreModule } from './mongoose-core.module';

@Module({})
export class MongooseModule extends NestMongooseModule {
	static forRoot(
		uri: string,
		options: MongooseModuleOptions = {},
	): DynamicModule {
		return {
			module: MongooseModule,
			imports: [MongooseCoreModule.forRoot(uri, options)],
		}
	}

	static forRootAsync(options: MongooseModuleAsyncOptions): DynamicModule {
		return {
			module: MongooseModule,
			imports: [MongooseCoreModule.forRootAsync(options)],
		}
	}
}

这是最外层，用于引入 MongooseCoreModule 模块, 重写有问题的源码

2 ）新建 src/database/mongoose/mongoose-core.module.ts

import {
  DynamicModule,
  Global,
  Inject,
  Module,
  OnApplicationShutdown,
  Provider,
  Type,
} from '@nestjs/common';
import { ModuleRef } from '@nestjs/core';
import * as mongoose from 'mongoose';
import { ConnectOptions, Connection } from 'mongoose';
import { defer, lastValueFrom } from 'rxjs';
import { catchError } from 'rxjs/operators';
import { handleRetry } from './mongoose.utils';
import {
  MONGOOSE_CONNECTION_NAME,
  MONGOOSE_MODULE_OPTIONS,
} from './mongoose.constants';

import {
  MongooseModuleOptions,
  MongooseModuleAsyncOptions,
  MongooseModuleFactoryOptions,
  MongooseOptionsFactory,
  getConnectionToken,
} from '@nestjs/mongoose';

@Global()
@Module({})
export class MongooseCoreModule implements OnApplicationShutdown {
  private static connections: Record<string, mongoose.Connection> = {};
  constructor(
    @Inject(MONGOOSE_CONNECTION_NAME) private readonly connectionName: string,
    private readonly moduleRef: ModuleRef,
  ) {}

  static forRoot(
    uri: string,
    options: MongooseModuleOptions = {},
  ): DynamicModule {
    const {
      retryAttempts,
      retryDelay,
      connectionName,
      connectionFactory,
      connectionErrorFactory,
      lazyConnection,
      onConnectionCreate,
      verboseRetryLog,
      ...mongooseOptions
    } = options;

    const mongooseConnectionFactory =
      connectionFactory || ((connection) => connection);

    const mongooseConnectionError =
      connectionErrorFactory || ((error) => error);

    const mongooseConnectionName = getConnectionToken(connectionName);

    const mongooseConnectionNameProvider = {
      provide: MONGOOSE_CONNECTION_NAME,
      useValue: mongooseConnectionName,
    };

    const connectionProvider = {
      provide: mongooseConnectionName,
      useFactory: async (): Promise<any> =>
        await lastValueFrom(
          defer(async () =>
            mongooseConnectionFactory(
              await this.createMongooseConnection(uri, mongooseOptions, {
                lazyConnection,
                onConnectionCreate,
              }),
              mongooseConnectionName,
            ),
          ).pipe(
            handleRetry(retryAttempts, retryDelay, verboseRetryLog),
            catchError((error) => {
              throw mongooseConnectionError(error);
            }),
          ),
        ),
    };
    return {
      module: MongooseCoreModule,
      providers: [connectionProvider, mongooseConnectionNameProvider],
      exports: [connectionProvider],
    };
  }

  static forRootAsync(options: MongooseModuleAsyncOptions): DynamicModule {
    const mongooseConnectionName = getConnectionToken(options.connectionName);

    const mongooseConnectionNameProvider = {
      provide: MONGOOSE_CONNECTION_NAME,
      useValue: mongooseConnectionName,
    };

    const connectionProvider = {
      provide: mongooseConnectionName,
      useFactory: async (
        mongooseModuleOptions: MongooseModuleFactoryOptions,
      ): Promise<any> => {
        const {
          retryAttempts,
          retryDelay,
          uri,
          connectionFactory,
          connectionErrorFactory,
          lazyConnection,
          onConnectionCreate,
          verboseRetryLog,
          ...mongooseOptions
        } = mongooseModuleOptions;

        const mongooseConnectionFactory =
          connectionFactory || ((connection) => connection);

        const mongooseConnectionError =
          connectionErrorFactory || ((error) => error);

        return await lastValueFrom(
          defer(async () =>
            mongooseConnectionFactory(
              await this.createMongooseConnection(
                uri as string,
                mongooseOptions,
                { lazyConnection, onConnectionCreate },
              ),
              mongooseConnectionName,
            ),
          ).pipe(
            handleRetry(retryAttempts, retryDelay, verboseRetryLog),
            catchError((error) => {
              throw mongooseConnectionError(error);
            }),
          ),
        );
      },
      inject: [MONGOOSE_MODULE_OPTIONS],
    };
    const asyncProviders = this.createAsyncProviders(options);
    return {
      module: MongooseCoreModule,
      imports: options.imports,
      providers: [
        ...asyncProviders,
        connectionProvider,
        mongooseConnectionNameProvider,
      ],
      exports: [connectionProvider],
    };
  }

  private static createAsyncProviders(
    options: MongooseModuleAsyncOptions,
  ): Provider[] {
    if (options.useExisting || options.useFactory) {
      return [this.createAsyncOptionsProvider(options)];
    }
    const useClass = options.useClass as Type<MongooseOptionsFactory>;
    return [
      this.createAsyncOptionsProvider(options),
      {
        provide: useClass,
        useClass,
      },
    ];
  }

  private static createAsyncOptionsProvider(
    options: MongooseModuleAsyncOptions,
  ): Provider {
    if (options.useFactory) {
      return {
        provide: MONGOOSE_MODULE_OPTIONS,
        useFactory: options.useFactory,
        inject: options.inject || [],
      };
    }
    // `as Type<MongooseOptionsFactory>` is a workaround for microsoft/TypeScript#31603
    const inject = [
      (options.useClass || options.useExisting) as Type<MongooseOptionsFactory>,
    ];
    return {
      provide: MONGOOSE_MODULE_OPTIONS,
      useFactory: async (optionsFactory: MongooseOptionsFactory) =>
        await optionsFactory.createMongooseOptions(),
      inject,
    };
  }

  private static async createMongooseConnection(
    uri: string,
    mongooseOptions: ConnectOptions,
    factoryOptions: {
      lazyConnection?: boolean;
      onConnectionCreate?: MongooseModuleOptions['onConnectionCreate'];
    },
  ): Promise<Connection> {
    // 添加这里
    if (this.connections[uri]) {
      return this.connections[uri];
    }

    const connection = mongoose.createConnection(uri, mongooseOptions);
    this.connections[uri] = connection; // 这里赋值


    if (factoryOptions?.lazyConnection) {
      return connection;
    }

    factoryOptions?.onConnectionCreate?.(connection);

    return connection.asPromise();
  }

  async onApplicationShutdown() {
    const connection = this.moduleRef.get<any>(this.connectionName);
    if (connection) {
      await connection.close();
    }
    const connectionClients = Object.values(MongooseCoreModule.connections);
    if (connectionClients.length > 0) {
      // 销毁所有 mongoose connection
      for (const client of connectionClients) {
        client?.close();
      }
    }
  }
}

这里，注意 createMongooseConnection 以及 onApplicationShutdown 中的处理
对连接进行优化处理，以及在异常关闭时对客户端进行销毁

3 ）src/database/mongoose/mongoose.constants.ts

// 这个配置模拟调接口/读数据库获取的
export const tenantMap = new Map([
	['1', 'T1'],
	['2', 'T2']
]);
export const defaultTenant = tenantMap.values().next().value; // 第一个

// 新增如下
export const DEFAULT_DB_CONNECTION = 'DatabaseConnection';
export const MONGOOSE_MODULE_OPTIONS = 'MongooseModuleOptions';
export const MONGOOSE_CONNECTION_NAME = 'MongooseConnectionName';

export const RAW_OBJECT_DEFINITION = 'RAW_OBJECT_DEFINITION';

3 ）src/database/mongoose/mongoose.utils.ts

import { Logger } from '@nestjs/common';
import { Observable } from 'rxjs';
import { delay, retryWhen, scan } from 'rxjs/operators';

export function handleRetry(
  retryAttempts = 9,
  retryDelay = 3000,
  verboseRetryLog = false,
): <T>(source: Observable<T>) => Observable<T> {
  const logger = new Logger('MongooseModule');
  return <T>(source: Observable<T>) =>
    source.pipe(
      retryWhen((e) =>
        e.pipe(
          scan((errorCount, error) => {
            const verboseMessage = verboseRetryLog
              ? ` Message: ${error.message}.`
              : '';
            const retryMessage =
              retryAttempts > 0 ? ` Retrying (${errorCount + 1})...` : '';

            logger.error(
              [
                'Unable to connect to the database.',
                verboseMessage,
                retryMessage,
              ].join(''),
              error.stack,
            );
            if (errorCount + 1 >= retryAttempts) {
              throw error;
            }
            return errorCount + 1;
          }, 0),
          delay(retryDelay),
        ),
      ),
    );
}

工具包的部分函数

4 ）测试

重启项目，连入其中一个mongo 的 docker 容器，进入数据库，执行 db.serverStatus().connections
调用对应数据库的租户标识的接口，再次执行 db.serverStatus().connections
可看到 current 和 active 增加了，后面多次调用同一租户接口则不会再增加
结束程序，则会销毁客户端，相应数字会同步减少
这样就完成了相关功能

总结

问题本质：Mongoose 在 NestJS 中的连接未复用，导致连接数异常增长
核心影响：IO 资源浪费、性能下降、数据库连接池耗尽
解决思路：
- 在服务层缓存已有连接
- 修改 Mongoose 模块源码逻辑
- 使用第三方连接池库进行封装
- 最佳实践：
  - 多租户系统中应确保数据库连接的唯一性与复用性
  - 建议对 Mongoose 模块进行轻量级封装以适配业务需求