Tech blog

  • Resume
  • Posts
  • About

๐Ÿ˜Štest ์ œ๋ชฉ

#tag1#tag2

2023-07-02

๊ธ€๋‚ด์šฉ์„ ์—ฌ๊ธฐ์— ์“ฐ๋Š”๊ฒ๋‹ˆ๋‹ค.

  • test

์†Œ์ œ๋ชฉ

์ด๋ฒˆ ํฌ์ŠคํŒ…์—์„œ๋Š” GraphQL ์—์„œ N+1 ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•œ ์†”๋ฃจ์…˜์ธ DataLoader์— ๋Œ€ํ•œ ์†Œ๊ฐœ์™€ GraphQL ์— DataLoader๋ฅผ ์–ด๋–ค์‹์œผ๋กœ ์ ์šฉํ•ด์•ผ๋˜๋Š”์ง€๋ฅผ ์ •๋ฆฌํ•ด๋ณด๋ ค๊ณ  ํ•œ๋‹ค.

N+1 ๋ฌธ์ œ

N+1 ๋ฌธ์ œ๋Š” ORM์„ ์‚ฌ์šฉํ• ๋•Œ ์ฃผ๋กœ ๋ฐœ์ƒํ•˜๋Š” ์„ฑ๋Šฅ ๋ฌธ์ œ์ด๋‹ค.

Post ์—”ํ‹ฐํ‹ฐ์™€ Comment ์—”ํ‹ฐํ‹ฐ๊ฐ€ ์žˆ๋‹ค๊ณ  ๊ฐ€์ •ํ•ด๋ณด์ž. ์ด ์—”ํ‹ฐํ‹ฐ๊ฐ„์˜ ๊ด€๊ณ„๋Š” 1:N์œผ๋กœ ์ •์˜ํ•  ์ˆ˜ ์žˆ๋‹ค. Post ๋ชฉ๋ก๊ณผ post์— ํ•ด๋‹นํ•˜๋Š” comments๋ฅผ ์กฐํšŒํ•˜๋ ค ํ•œ๋‹ค๋ฉด comment entity๊ฐ€ lazy loading๋˜๋ฉด์„œ N(๊ฐ๊ฐ์˜ comments ์กฐํšŒ) + 1(post ์กฐํšŒ) ๋งŒํผ ์ฟผ๋ฆฌ๊ฐ€ ์‹คํ–‰๋˜์„œ N+1 ๋ฌธ์ œ๋ผ๊ณ  ๋ถ€๋ฅธ๋‹ค.

ํ•ด๊ฒฐ๋ฐฉ๋ฒ•์œผ๋กœ๋Š” lazy loading์„ ํ•˜๋Š” ๋Œ€์‹  ๋ฏธ๋ฆฌ JOIN ์—ฐ์‚ฐ์„ ํ†ตํ•ด fetchํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์ƒ๊ฐํ•  ์ˆ˜ ์žˆ๋‹ค. ์—ฌ๊ธฐ์— ๋Œ€ํ•œ ๋ฐฉ๋ฒ•์€ ORM ๋งˆ๋‹ค ์ฐจ์ด๊ฐ€ ์žˆ์„ ์ˆ˜ ์žˆ๋‹ค.

// TypeORM example // lazy const posts = await Post.find({}) const comments = await Promise.all( posts.map(async p => { // TypeORM์—์„œ lazy relation ๊ฒฝ์šฐ p.comments ๋Š” promise array ์ด๋‹ค. const comments = await p.comments return comments }), ) // eager const posts = await Post.find({ relations: ['comments'] })

GraphQL ์—์„œ์˜ N+1 ๋ฌธ์ œ

์—์„œ ๋ฐœ์ƒํ•˜๋Š” N+1๋ฌธ์ œ๋„ ์œ„์™€ ๋น„์Šทํ•˜๋‹ค. ์œ„์—์„œ์˜ ์—”ํ‹ฐํ‹ฐ๊ด€๊ณ„๋ฅผ GraphQL SDL๋กœ ์ž‘์„ฑํ•˜๋ฉด ์•„๋ž˜์™€ ๊ฐ™๋‹ค.

type Post { id: Int! title: String! content: String! comments: [Comment!]! } type Comment { id: Int! content: String! } type Query { posts: [Post!]! }

post ๋ชฉ๋ก๊ณผ comments ๋ฅผ ๊ฐ€์ ธ์˜ค๋Š” query๋ฅผ ์ž‘์„ฑํ•ด๋ณด์ž.

query { posts { # posts query (1) id title content comments { # comments query (N) -> Post ๊ฐœ์ˆ˜๋งŒํผ id ... } } }

์œ„์™€ ๊ฐ™์ด comments ๋ถ€๋ถ„์—์„œ ์„ฑ๋Šฅ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•˜๊ฒŒ ๋œ๋‹ค. GraphQL ์—์„œ resolver๋ฅผ ๊ตฌ์„ฑํ•˜๋Š” ์ „๋žต์€ ์—ฌ๋Ÿฌ๊ฐ€์ง€๊ฐ€ ์žˆ๊ฒ ์ง€๋งŒ ๋ณดํ†ต ๋‹ค๋ฅธ type๊ณผ ๊ด€๊ณ„๊ฐ€ ์žˆ๋Š” ๊ฒฝ์šฐ resolver๋ฅผ ๋ถ„๋ฆฌํ•ด์„œ ๊ตฌํ˜„ํ•˜๋Š” ๊ฒฝ์šฐ๊ฐ€ ๋งŽ๊ธฐ ๋•Œ๋ฌธ์ด๋‹ค.

const resolver = { Query: { posts: async () => { return Post.find({}) }, }, Post: { // posts query๋ฅผ ํ˜ธ์ถœํ• ๊ฒฝ์šฐ post ๊ฐœ์ˆ˜(N) ๋งŒํผ ํ˜ธ์ถœ๋œ๋‹ค. comments: async root => { return Comment.find({ where: { postId: root.id } }) }, }, }

๋ฌผ๋ก  ์•„๋ž˜์™€ ๊ฐ™์ด posts resolver์—์„œ data๋ฅผ join ํ•ด์„œ fetchํ›„์— return ํ•œ๋‹ค๋ฉด N+1 ๋ฌธ์ œ๋Š” ๋ฐœ์ƒํ•˜์ง€ ์•Š์„ ์ˆ˜ ์žˆ๋‹ค.

const resovler = { Query: { posts: async () => { const posts = await Post.find({ relations: ['comments'] }) return posts }, }, } /* query { posts { id title content } } */

ํ•˜์ง€๋งŒ ์œ„์™€ ๊ฐ™์ด resolver๋ฅผ ๊ตฌํ˜„ํ•œ๋‹ค๋ฉด query์—์„œ comments field ๋ฅผ ์š”์ฒญํ•˜์ง€ ์•Š์•„๋„ joinํ•ด์„œ fetchํ•ด์„œ data๋ฅผ ๊ฐ€์ ธ์˜ค๊ฒŒ๋œ๋‹ค. client์—์„œ ๋ฐ›๋Š” ๋ฐ์ดํ„ฐ๋Š” over fetching์ด ์ผ์–ด๋‚˜์ง€ ์•Š์ง€๋งŒ ์‹ค์ œ ๋ฐ์ดํ„ฐ๋ฅผ loadํ•˜๋Š” ๊ณณ์—์„œ๋Š” over fetching์ด ์ผ์–ด๋‚˜๊ณ  ์žˆ๋Š” ๊ฒƒ์ด๋‹ค.

DataLoader

DataLoader๋Š” data fetch ํ• ๋•Œ ๋‚˜ํƒ€๋‚˜๋Š” N+1 ๋ฌธ์ œ๋ฅผ batching์„ ํ†ตํ•ด 1+1๋กœ ๋ณ€ํ™˜ํ•ด์ฃผ๋Š” library์ด๋‹ค. ์ฃผ๋กœ ์—์„œ ๋งŽ์ด ์‚ฌ์šฉ๋˜์ง€๋งŒ GrpahQL์— ์–ด๋–ค ์˜์กด์„ฑ์„ ๊ฐ€์ง€๊ณ  ์žˆ์ง€๋Š” ์•Š๋‹ค.

Batching

DataLoader๋Š” javascript์˜ event-loop ์„ ์ด์šฉํ•œ๋‹ค. ์ฃผ์š”๊ธฐ๋Šฅ์ธ batching์€ event-loop ์ค‘ ํ•˜๋‚˜์˜ tick์—์„œ ์‹คํ–‰๋œ data fetch์— ๋Œ€ํ•œ ์š”์ฒญ์„ ํ•˜๋‚˜์˜ ์š”์ฒญ์œผ๋กœ ๋ชจ์•„์„œ ์‹คํ–‰ํ•˜๊ณ  ๊ทธ ๊ฒฐ๊ณผ๋ฅผ ๋‹ค์‹œ ์•Œ๋งž๊ฒŒ ๋ถ„๋ฐฐํ•˜๋Š” ์—ญํ• ์„ ํ•œ๋‹ค.

์•„๋ž˜ ๊ฐ„๋‹จํ•œ ์˜ˆ์ œ๋ฅผ ๋ณด์ž.

const DataLoader = require('dataloader') // fake data const posts = [ { id: 1, title: 'test1' }, { id: 2, title: 'test2' }, { id: 3, title: 'test3' }, { id: 4, title: 'test4' }, { id: 5, title: 'test5' }, ] // fake db operation const findAllPosts = () => new Promise(resolve => { setTimeout(() => { resolve(posts) }, 100) }) // batchLoadFn ์˜ ๊ฒฐ๊ณผ๋Š” promise์—ฌ์•ผ ํ•œ๋‹ค. const batchLoadFn = async keys => { const results = await findAllPosts() console.log(keys) // db ์—์„œ ๋ฐ›์•„์˜จ ๊ฒฐ๊ณผ๋ฅผ ์š”์ฒญ์˜จ key์— mapping return keys.map(k => results.find(p => p.id === k)) } const postLoader = new DataLoader(batchLoadFn) // tick 1 postLoader.load(1).then(console.log) postLoader.load(2).then(console.log) // tick 2 setTimeout(() => { postLoader.load(3).then(console.log) postLoader.load(4).then(console.log) }, 100)

DataLoader์˜ constructor๋Š” batch์š”์ฒญ์„ ์–ด๋–ป๊ฒŒ ์ฒ˜๋ฆฌํ• ์ง€์— ๋Œ€ํ•œ batchLoadFn ์„ ์ธ์ž๋กœ ๋ฐ›๋Š”๋‹ค. ์ด batchLoadFn์˜ ์—ญํ• ์€ ํ•˜๋‚˜์˜ tick์—์„œ ๋“ค์–ด์˜จ key๋“ค์— ๋Œ€ํ•œ ์š”์ฒญ์„ ๋ชจ์•„์„œ ํ•˜๋‚˜์˜ ์š”์ฒญ์„ ๋งŒ๋“ค์–ด DB์— queryํ•˜๊ณ  ๊ทธ ๊ฒฐ๊ณผ๋ฅผ ์š”์ฒญ์˜จ key์— ๋งž๊ฒŒ mapping ํ•œ๋‹ค. (์ด ์˜ˆ์ œ์—์„œ๋Š” ํŽธ์˜์ƒ memory์ƒ์— data๋ฅผ ํ™œ์šฉํ•˜์˜€๋‹ค.)

setTimeout์€ event-loop ์ƒ์— ํ•˜๋‚˜์˜ tick ์—์„œ ์‹คํ–‰๋˜์ง€ ์•Š๊ณ  ๋‹ค์Œ์œผ๋กœ ์‹คํ–‰์„ ๋ฏธ๋ฃจ๊ฒŒ ๋œ๋‹ค.

Output

[1, 2] [3, 4] { id: 1 ... } { id: 2 ... } { id: 3 ... } { id: 4 ... }

tick ์ด 2๋ฒˆ ๋ฐœ์ƒํ–ˆ๊ธฐ ๋•Œ๋ฌธ์— load๋ฅผ 4๋ฒˆ ํ˜ธ์ถœํ•˜์—ฌ๋„ ์‹ค์ œ์š”์ฒญ์€ 2๋ฒˆ๋งŒ ์‹คํ–‰๋˜๋Š”๊ฑธ ํ™•์ธํ•  ์ˆ˜ ์žˆ๋‹ค.

์ „์ฒด์ ์ธ ๋™์ž‘์„ ๋‹ค์‹œ ์ •๋ฆฌํ•˜๋ฉด load ๋ฅผ ๊ฐœ๋ณ„์ ์œผ๋กœ ํ˜ธ์ถœํ•˜์ง€๋งŒ ์‹คํ–‰๋˜๋Š” tick ๋ณ„๋กœ groupingํ•ด์„œ batch ์š”์ฒญ์„ ํ•˜๊ฒŒ๋˜๊ณ  ๊ทธ ๊ฒฐ๊ณผ๋ฅผ ๋‹ค์‹œ ๊ฐœ๋ณ„์ ์œผ๋กœ ๋‚˜๋ˆ ์„œ ๋ฐ˜ํ™˜ํ•˜๊ฒŒ ๋œ๋‹ค. ์ฆ‰, ๋ฐ์ดํ„ฐ๋ฅผ lazy loadingํ•˜๋ฉด์„œ ์„ฑ๋Šฅ์ €ํ•˜์˜ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ๋‹ค.

GraphQL์— DataLoader ์ ์šฉ

์•„๊นŒ ๋ฌธ์ œ๊ฐ€ ๋˜์—ˆ๋˜ resolver์— DataLoader๋ฅผ ์ ์šฉํ•ด๋ณด์ž. comments ์—์„œ data๋ฅผ fetchingํ•˜๋Š” ๋ถ€๋ถ„์— DataLoader๋ฅผ ์ ์šฉํ•ด์„œ ํ•ด๊ฒฐํ•  ์ˆ˜ ์žˆ๋‹ค.

GraphQL ์—์„œ DataLoader๋ฅผ ์ ์šฉํ•˜๋Š” ์ˆœ์„œ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.

  1. loadํ•  data์— ๋”ฐ๋ผ batchLoadFn ๋ฅผ ์ž‘์„ฑํ•œ๋‹ค.
  2. Context์—์„œ ํ•ด๋‹น DataLoader ๊ฐ์ฒด๋ฅผ ์ƒ์„ฑํ•œ๋‹ค.
  3. resolver์—์„œ context์˜ DataLoader๋ฅผ ํ†ตํ•ด์„œ load๋ฅผ ํ˜ธ์ถœํ•œ๋‹ค.

DataLoader์˜ instance๋Š” ์ž์ฒด์ ์œผ๋กœ cacheMap ์„ ๊ฐ€์ง€๊ณ  ์žˆ๋‹ค. ๊ฐ™์€ key์— ๋Œ€ํ•œ ์š”์ฒญ์ด ๋“ค์–ด์˜ค๋ฉด caching๋œ ๊ฐ’์„ ์‚ฌ์šฉํ•˜๊ฒŒ ๋˜๋Š”๋ฐ web application์—์„œ ์ด๋Ÿฐ๋ฐฉ์‹์€ ์œ„ํ—˜ํ•  ์ˆ˜ ์žˆ๋‹ค. ์ด๋Ÿฌํ•œ ์ด์œ ๋กœ ๋งค request๋งˆ๋‹ค ์ƒˆ๋กœ์šด DataLoader ๊ฐ์ฒด๋ฅผ ์ƒ์„ฑํ•ด์„œ ์‚ฌ์šฉํ•˜๋Š”๊ฒƒ์„ ๊ถŒ์žฅํ•˜๊ณ  ์žˆ๋‹ค.

CommentsLoader

const batchLoadFn = async postIds => { const comments = await Comment.find({ where: { postId: In(postIds) } }) return postIds.map(id => comments.filter(c => c.postId === id)) } export const commentsLoader = () => new DataLoader(batchLoadFn)

Context

const server = new ApolloServer({ ..., context: () => ({ loaders: { commentsLoader: commentsLoader() } }) })

Resolver

... comments: async (root, _, context) => { return context.loaders.commentsLoader.load(root.id) }

์œ„์™€๊ฐ™์ด DataLoader๋ฅผ ์ ์šฉํ•˜๋ฉด comments resolver์—์„œ์˜ ๋ชจ๋“  load ์š”์ฒญ์€ ํ•˜๋‚˜์˜ ์š”์ฒญ์œผ๋กœ ๋ฌถ์—ฌ์„œ ์‹คํ–‰๋˜์„œ ์„ฑ๋Šฅ๋ฌธ์ œ๊ฐ€ ํ•ด๊ฒฐ๋œ๋‹ค. ๋˜ํ•œ data๋ฅผ ์ดˆ๊ธฐ์— ๋ชจ๋‘ fetchํ•˜์ง€ ์•Š๊ณ  lazyํ•˜๊ฒŒ ์œ ์ง€ํ•  ์ˆ˜ ์žˆ๊ธฐ ๋•Œ๋ฌธ์— ์—ญํ• ์— ๋งž๊ฒŒ resolver๋ฅผ ๋ถ„๋ฆฌํ•ด์„œ ๋ณต์žก์„ฑ์„ ์ค„์ผ ์ˆ˜ ์žˆ๋‹ค.

๋งˆ์น˜๋ฉฐ

DataLoader๋ฅผ ์ ์šฉํ•˜๋Š” ๊ฒƒ์€ ํฐ ์–ด๋ ค์›€์ด ์—†์—ˆ๋˜ ๊ฒƒ ๊ฐ™๋‹ค. ์ฒ˜์Œ์—” ๋™์ž‘๋ฐฉ์‹์ด ์ข€ ๋‚œํ•ดํ•˜๊ฒŒ ๋Š๊ปด์กŒ๋Š”๋ฐ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ ์ฝ”๋“œ๋ฅผ ์ฝ๊ณ ๋‚˜์„œ ๋ณด๋‹ˆ Promise์˜ ์ด์ ๊ณผ event-loop์˜ ํŠน์„ฑ์„ ์ด์šฉํ•˜๋Š” ๋ถ€๋ถ„์ด ์ธ์ƒ์ ์ด์˜€๋‹ค.

Ref

  • https://github.com/graphql/dataloader
  • https://medium.com/gaplabs-engineering/make-more-efficient-requests-with-dataloader-96ff50eb8998
  • https://nodejs.org/ko/docs/guides/event-loop-timers-and-nexttick/
  • https://engineering.shopify.com/blogs/engineering/solving-the-n-1-problem-for-graphql-through-batching
  • https://github.com/typeorm/typeorm/blob/master/docs/eager-and-lazy-relations.md