LangChain LCEL のbatchの並列処理パラメータ
挙動を見るに、指定しなければ256並列(何かのリミット?)っぽく動き、chain.batch(targets, config={"max_concurrency": 100}) のように指定することで、並列で実行する。
ええ、この並列ってchain毎に並列実行じゃなくて、Runnableごとに一度止まるの!思った挙動と違って使いにくいな・・・。
code:python
from langchain_core.runnables import chain as to_chain
import time
@to_chain
def tap_sleep_100msec(data):
time.sleep(0.1)
return data
targets = list(range(1000))
start = time.time()
result = tap_sleep_100msec.batch(targets)
end = time.time()
print(f"time no max_concurrency {end - start}")
start = time.time()
result = tap_sleep_1sec.batch(targets, config={"max_concurrency": 100})
end = time.time()
print(f"time with max_concurrency=10 -> {end - start}")
# time no max_concurrency 3.221008539199829
# time with max_concurrency=10 -> 10.0453462600708
code:python
from langchain_core.runnables import chain as to_chain
import time
@to_chain
def _1st_tap_sleep_1sec(data):
time.sleep(1)
print("1st!")
return data
@to_chain
def _2nd_tap(data):
time.sleep(1)
print("2nd")
return data
targets = list(range(5))
chain = _1st_tap_sleep_1sec | _2nd_tap
start = time.time()
result = chain.batch(targets)
end = time.time()
print(f"time no max_concurrency {end - start}")
start = time.time()
result = chain.batch(targets, config={"max_concurrency": 3})
end = time.time()
print(f"time with max_concurrency=3 -> {end - start}")
結果。最初のrunnableが終わるまで待ちが発生して、その後次のrunnableへ。まじか。
code:txt
1st!
1st!
1st!
1st!
1st!
2nd2nd
2nd
2nd
2nd
time no max_concurrency 2.013137102127075
1st!
1st!
1st!
1st!
1st!
2nd2nd
2nd
2nd
2nd
time with max_concurrency=3 -> 4.0129876136779785