Skip to content

torch.compile 教程

译者:片刻小哥哥

项目地址:https://pytorch.apachecn.org/2.0/tutorials/intermediate/torch_compile_tutorial

原始地址:https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html

作者: 文威廉

torch.compile 是加速 PyTorch 代码速度的最新方法! torch.compile 通过将 PyTorch 代码 JIT 编译到优化的内核中,使 PyTorch 代码运行得更快, 同时需要最少的资源代码更改。

在本教程中,我们将介绍 torch.compile 的基本用法, 并演示 torch.compile 相对于 以前的 PyTorch 编译器解决方案的优势,例如 TorchScriptFX 跟踪 .

内容

所需的 pip 依赖项

  • `torch

=

2.0*torchvision*numpy*scipy*制表`

注意:本教程建议使用现代 NVIDIA GPU(H100、A100 或 V100), 以便重现下面所示并在其他地方记录的加速数字。

import torch
import warnings

gpu_ok = False
if torch.cuda.is_available():
    device_cap = torch.cuda.get_device_capability()
    if device_cap in ((7, 0), (8, 0), (9, 0)):
        gpu_ok = True

if not gpu_ok:
    warnings.warn(
        "GPU is not NVIDIA V100, A100, or H100. Speedup numbers may be lower "
        "than expected."
    )
/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py:48: UserWarning:

GPU is not NVIDIA V100, A100, or H100. Speedup numbers may be lower than expected.

基本用法

torch.compile 包含在最新的 PyTorch 中。 在 GPU 上运行 TorchInductor 需要 Triton,它包含在 PyTorch 2.0 nightly 二进制文件中。如果 Triton 仍然丢失,请尝试通过 pip 安装 torchtriton

( `pip

install

torchtriton

--extra-index- url

"https://download.pytorch.org/whl/nightly/cu117"` 适用于 CUDA 11.7)。

可以通过将可调用函数传递给 torch.compile 来优化任意 Python 函数。然后我们可以调用返回的优化 函数来代替原始函数。

def foo(x, y):
    a = torch.sin(x)
    b = torch.cos(y)
    return a + b
opt_foo1 = torch.compile(foo)
print(opt_foo1(torch.randn(10, 10), torch.randn(10, 10)))
tensor([[ 1.6850,  1.9924,  1.7090,  0.0034,  1.1414, -0.1822,  0.4861, -0.0536,
         -0.2252,  1.9398],
        [ 0.3693, -0.0695,  0.1748,  0.3436,  0.1939,  1.5721,  1.9882, -0.2235,
          0.3161,  1.2642],
        [ 0.2480,  1.8793,  1.7152,  1.6772,  1.8881,  1.4748,  1.3466,  1.7763,
          0.7469,  1.0407],
        [-0.1121,  1.6015, -0.0188,  0.2128,  0.5218,  1.9838,  0.8185,  0.5093,
         -0.3603,  0.1793],
        [-1.7890,  1.7532, -0.4040,  0.1222, -0.0029,  1.7975, -0.3877,  0.5123,
          0.1673,  0.1330],
        [ 1.0627,  0.9609,  0.1019,  1.8814,  0.1142, -0.2338, -0.9621,  0.7631,
          0.6506,  0.1853],
        [ 0.4584,  1.7648, -0.0444,  1.9610,  1.5884,  0.7353,  1.2190,  1.3662,
          1.0938, -0.1587],
        [-0.7502,  1.6640,  0.3495,  1.3496,  0.8187,  1.1719,  0.5820,  0.1498,
          0.0885,  0.1036],
        [ 0.3961,  0.6043, -0.0861, -0.3371,  0.8622,  1.4341,  1.2988,  0.5023,
          0.3074,  0.1277],
        [ 0.9748,  0.4117,  1.2616,  1.6314,  0.4693,  0.4092,  0.0401,  1.1196,
          1.2458,  1.3280]])

或者,我们可以修饰该函数。

@torch.compile
def opt_foo2(x, y):
    a = torch.sin(x)
    b = torch.cos(y)
    return a + b
print(opt_foo2(torch.randn(10, 10), torch.randn(10, 10)))
tensor([[ 0.5360,  0.1697, -0.0561,  0.1890, -0.1310,  1.2276,  1.1739,  0.1944,
         -0.1561,  1.6990],
        [ 1.0421,  1.9472,  0.2682,  0.2701,  1.3346,  0.7651,  1.0897,  1.1730,
          0.6161,  0.9223],
        [ 1.5756,  1.5294,  0.0112, -0.1522, -0.7674,  1.8515, -0.2443,  0.3696,
          0.2693,  0.8735],
        [-0.3701,  1.1190,  1.4164,  1.8648,  1.2080,  0.0732,  1.5274,  0.6868,
          1.2440,  1.0715],
        [-1.2454, -0.0159,  0.4315,  0.1317,  1.0530, -1.0603, -0.0532,  0.6661,
          1.7101, -0.2076],
        [-0.7091,  0.7824,  1.7161,  1.2750,  0.6368,  1.2488,  0.4897,  1.2429,
          1.3409,  1.3735],
        [ 0.8345,  0.0653,  0.3462,  1.2383, -0.4092,  1.6438, -0.0962,  0.4011,
          0.2463, -0.5802],
        [ 1.6349,  0.7297,  1.2547, -0.3113,  0.9310,  0.1162,  1.7618,  0.4882,
          0.7640,  0.2930],
        [ 1.1669, -0.7775,  1.2000,  0.6008, -0.2814,  0.5541,  0.5753,  1.4731,
          1.6835,  0.7370],
        [ 1.5087,  0.6195,  0.1153,  1.2966,  1.8815,  1.1678,  1.5686,  1.6018,
          0.2193,  1.3500]])

我们还可以优化 torch.nn.Module 实例。

class MyModule(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.lin = torch.nn.Linear(100, 10)

    def forward(self, x):
        return torch.nn.functional.relu(self.lin(x))

mod = MyModule()
opt_mod = torch.compile(mod)
print(opt_mod(torch.randn(10, 100)))
tensor([[0.0000, 0.0000, 0.2419, 0.0446, 0.9011, 0.2674, 0.3633, 0.4984, 0.0000,
         0.0988],
        [0.6906, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.8490, 0.0000, 0.0000,
         0.5475],
        [0.0852, 0.2762, 0.7441, 0.0000, 0.0000, 0.1820, 0.0000, 0.0000, 0.0000,
         0.0334],
        [0.3024, 0.0077, 1.2572, 0.0000, 0.0000, 0.6520, 0.0000, 0.0000, 0.0000,
         0.8976],
        [0.1998, 0.3333, 0.0000, 0.7803, 0.4202, 0.0915, 0.0000, 1.2543, 0.0000,
         0.4615],
        [0.2487, 0.4187, 0.0000, 0.0000, 0.5124, 0.0000, 0.2512, 0.0000, 0.5850,
         0.0000],
        [0.0000, 0.0048, 0.0000, 0.0000, 0.0000, 0.2287, 0.0000, 0.4841, 0.3915,
         0.0000],
        [0.2017, 0.0000, 0.0896, 1.4135, 0.0593, 0.3788, 0.0000, 0.0000, 0.0000,
         0.4972],
        [0.0000, 0.0000, 1.6580, 0.6414, 0.0000, 0.0000, 0.0000, 0.0000, 0.6491,
         0.7755],
        [0.0000, 0.0000, 0.6442, 0.0260, 0.7456, 0.1000, -0.0000, -0.0000, 0.5366,
         0.1193]], grad_fn=<CompiledFunctionBackward>)

演示加速

现在让’s 演示使用 torch.compile 可以加速 真实模型。我们将通过在随机数据上评估和训练 torchvision 模型来比较标准 Eager 模式和 torch.compile

在开始之前,我们需要定义一些实用函数。

# Returns the result of running `fn()` and the time it took for `fn()` to run,
# in seconds. We use CUDA events and synchronization for the most accurate
# measurements.
def timed(fn):
    start = torch.cuda.Event(enable_timing=True)
    end = torch.cuda.Event(enable_timing=True)
    start.record()
    result = fn()
    end.record()
    torch.cuda.synchronize()
    return result, start.elapsed_time(end) / 1000

# Generates random input and targets data for the model, where `b` is
# batch size.
def generate_data(b):
    return (
        torch.randn(b, 3, 128, 128).to(torch.float32).cuda(),
        torch.randint(1000, (b,)).cuda(),
    )

N_ITERS = 10

from torchvision.models import densenet121
def init_model():
    return densenet121().to(torch.float32).cuda()

首先让’s 进行比较推理。

请注意,在调用 torch.compile 时,我们有额外的 mode 参数,我们将在下面讨论。

model = init_model()

# Reset since we are using a different mode.
import torch._dynamo
torch._dynamo.reset()

model_opt = torch.compile(model, mode="reduce-overhead")

inp = generate_data(16)[0]
with torch.no_grad():
    print("eager:", timed(lambda: model(inp))[1])
    print("compile:", timed(lambda: model_opt(inp))[1])
eager: 0.35339263916015623
/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_inductor/compile_fx.py:135: UserWarning:

TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled. Consider setting `torch.set_float32_matmul_precision('high')` for better performance.

compile: 53.191453125

请注意, torch.compile 与 eager 相比 需要更长的时间才能完成。这是因为 torch.compile 在执行时将模型编译为优化的内核。在我们的示例中,模型的结构’ 没有改变,因此不需要 重新编译。因此,如果我们多次运行优化后的模型, 与 eager 相比,我们应该会看到显着的改进。

eager_times = []
for i in range(N_ITERS):
    inp = generate_data(16)[0]
    with torch.no_grad():
        _, eager_time = timed(lambda: model(inp))
    eager_times.append(eager_time)
    print(f"eager eval time {i}: {eager_time}")

print("~" * 10)

compile_times = []
for i in range(N_ITERS):
    inp = generate_data(16)[0]
    with torch.no_grad():
        _, compile_time = timed(lambda: model_opt(inp))
    compile_times.append(compile_time)
    print(f"compile eval time {i}: {compile_time}")
print("~" * 10)

import numpy as np
eager_med = np.median(eager_times)
compile_med = np.median(compile_times)
speedup = eager_med / compile_med
assert(speedup > 1)
print(f"(eval) eager median: {eager_med}, compile median: {compile_med}, speedup: {speedup}x")
print("~" * 10)
eager eval time 0: 0.017328128814697266
eager eval time 1: 0.01571020793914795
eager eval time 2: 0.015571968078613281
eager eval time 3: 0.015590399742126464
eager eval time 4: 0.015550463676452637
eager eval time 5: 0.015865856170654297
eager eval time 6: 0.016306175231933593
eager eval time 7: 0.016372735977172852
eager eval time 8: 0.015467519760131837
eager eval time 9: 0.015433728218078613
~~~~~~~~~~
compile eval time 0: 0.5103267822265625
compile eval time 1: 0.0069550080299377445
compile eval time 2: 0.006748159885406494
compile eval time 3: 0.006738944053649902
compile eval time 4: 0.006744063854217529
compile eval time 5: 0.006743040084838867
compile eval time 6: 0.006758399963378906
compile eval time 7: 0.006740992069244385
compile eval time 8: 0.006755328178405762
compile eval time 9: 0.00673689603805542
~~~~~~~~~~
(eval) eager median: 0.015650303840637207, compile median: 0.006746111869812011, speedup: 2.3198998389976775x
~~~~~~~~~~

事实上,我们可以看到使用 torch.compile 运行我们的模型会带来显着的加速。加速主要来自于减少 Python 开销和 GPU 读/写,因此观察到的加速可能会因模型架构和批量大小等因素而有所不同。例如,如果模型’s 架构很简单 并且数据量很大,则瓶颈将是 GPU 计算,并且观察到的加速可能不太显着。

您还可能会看到不同的加速结果,具体取决于所选 mode 参数。 “减少开销”模式使用 CUDA 图来进一步减少 Python 的开销。对于您自己的模型, 您可能需要尝试不同的模式以最大限度地提高速度。您可以 在此处了解有关模式的更多信息 。

您可能还注意到,我们第二次使用 torch.compile 运行模型的速度明显 低于其他运行,尽管它比第一次运行快得多。这是因为 “减少开销” 模式为 CUDA 图运行一些预热迭代。

对于一般 PyTorch 基准测试,您可以尝试使用 torch.utils.benchmark 而不是我们上面定义的 timed 函数。我们在本教程中编写了自己的计时函数来显示 torch.compile ’s 编译延迟。

现在,让’s 考虑比较训练。

model = init_model()
opt = torch.optim.Adam(model.parameters())

def train(mod, data):
    opt.zero_grad(True)
    pred = mod(data[0])
    loss = torch.nn.CrossEntropyLoss()(pred, data[1])
    loss.backward()
    opt.step()

eager_times = []
for i in range(N_ITERS):
    inp = generate_data(16)
    _, eager_time = timed(lambda: train(model, inp))
    eager_times.append(eager_time)
    print(f"eager train time {i}: {eager_time}")
print("~" * 10)

model = init_model()
opt = torch.optim.Adam(model.parameters())
train_opt = torch.compile(train, mode="reduce-overhead")

compile_times = []
for i in range(N_ITERS):
    inp = generate_data(16)
    _, compile_time = timed(lambda: train_opt(model, inp))
    compile_times.append(compile_time)
    print(f"compile train time {i}: {compile_time}")
print("~" * 10)

eager_med = np.median(eager_times)
compile_med = np.median(compile_times)
speedup = eager_med / compile_med
assert(speedup > 1)
print(f"(train) eager median: {eager_med}, compile median: {compile_med}, speedup: {speedup}x")
print("~" * 10)
eager train time 0: 0.3766794128417969
eager train time 1: 0.05103615951538086
eager train time 2: 0.049339393615722656
eager train time 3: 0.04890419387817383
eager train time 4: 0.04913049697875976
eager train time 5: 0.04965273666381836
eager train time 6: 0.049137664794921876
eager train time 7: 0.04913356781005859
eager train time 8: 0.04952883148193359
eager train time 9: 0.049159168243408206
~~~~~~~~~~
skipping cudagraphs due to input mutation
compile train time 0: 282.300625
compile train time 1: 3.099315185546875
compile train time 2: 0.04109619140625
compile train time 3: 0.031270912170410156
compile train time 4: 0.03125657653808594
compile train time 5: 0.031254528045654296
compile train time 6: 0.03112550354003906
compile train time 7: 0.0311910400390625
compile train time 8: 0.03118182373046875
compile train time 9: 0.03119820785522461
~~~~~~~~~~
(train) eager median: 0.04924928092956543, compile median: 0.03125555229187012, speedup: 1.575697030392122x
~~~~~~~~~~

同样,我们可以看到 torch.compile 在第一次 迭代中需要更长的时间,因为它必须编译模型,但在随后的迭代中, 与 eager 相比,我们看到 显着的加速。

我们指出,本教程中提供的加速数字仅用于演示目的。官方加速值可在 TorchInductor 性能仪表板 查看 。

与 TorchScript 和 FX Tracing 的比较

我们已经看到 torch.compile 可以加速 PyTorch 代码。 为什么我们应该使用 torch.compile 而不是现有的 PyTorch 编译器解决方案,例如 TorchScript 或 FX Tracing? torch.compile 的 优势主要在于它能够处理 任意 Python 代码,并且对现有代码进行最少的更改。

torch.compile 可以处理其他编译器 解决方案难以解决的一种情况是数据相关的控制流( `if

x.sum()

<

0:` 下面一行)。

def f1(x, y):
    if x.sum() < 0:
        return -y
    return y

# Test that `fn1` and `fn2` return the same result, given
# the same arguments `args`. Typically, `fn1` will be an eager function
# while `fn2` will be a compiled function (torch.compile, TorchScript, or FX graph).
def test_fns(fn1, fn2, args):
    out1 = fn1(*args)
    out2 = fn2(*args)
    return torch.allclose(out1, out2)

inp1 = torch.randn(5, 5)
inp2 = torch.randn(5, 5)

TorchScript 跟踪 f1 会导致 错误的结果,因为 只跟踪实际的控制流路径。

traced_f1 = torch.jit.trace(f1, (inp1, inp2))
print("traced 1, 1:", test_fns(f1, traced_f1, (inp1, inp2)))
print("traced 1, 2:", test_fns(f1, traced_f1, (-inp1, inp2)))
/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py:274: TracerWarning:

Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!

traced 1, 1: True
traced 1, 2: False

FX 跟踪 f1 由于存在依赖于数据的控制流而导致错误。

import traceback as tb
try:
    torch.fx.symbolic_trace(f1)
except:
    tb.print_exc()
Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py", line 304, in <module>
    torch.fx.symbolic_trace(f1)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 1150, in symbolic_trace
    graph = tracer.trace(root, concrete_args)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 328, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 17, in inner
    return fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 817, in trace
    (self.create_arg(fn(*args)),),
  File "/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py", line 274, in f1
    if x.sum() < 0:
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/proxy.py", line 437, in __bool__
    return self.tracer.to_bool(self)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/proxy.py", line 300, in to_bool
    raise TraceError('symbolically traced variables cannot be used as inputs to control flow')
torch.fx.proxy.TraceError: symbolically traced variables cannot be used as inputs to control flow

如果我们在尝试 FX 跟踪 f1 时为 x 提供一个值,那么 我们会遇到与 TorchScript 跟踪相同的问题,因为数据相关的 控制流在跟踪函数中被删除。

fx_f1 = torch.fx.symbolic_trace(f1, concrete_args={"x": inp1})
print("fx 1, 1:", test_fns(f1, fx_f1, (inp1, inp2)))
print("fx 1, 2:", test_fns(f1, fx_f1, (-inp1, inp2)))
/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py:634: UserWarning:

Was not able to add assertion to guarantee correct input x to specialized function. It is up to the user to make sure that your inputs match the inputs you specialized the function with.

fx 1, 1: True
fx 1, 2: False

现在我们可以看到 torch.compile 正确处理 依赖于数据的控制流。

# Reset since we are using a different mode.
torch._dynamo.reset()

compile_f1 = torch.compile(f1)
print("compile 1, 1:", test_fns(f1, compile_f1, (inp1, inp2)))
print("compile 1, 2:", test_fns(f1, compile_f1, (-inp1, inp2)))
print("~" * 10)
compile 1, 1: True
compile 1, 2: True
~~~~~~~~~~

TorchScript 脚本可以处理依赖于数据的控制流,但是这个 解决方案有其自身的一系列问题。也就是说,TorchScript 脚本可能需要进行重大代码更改,并且在使用不支持的 Python 时会引发错误。

在下面的示例中,我们忘记了 TorchScript 类型注释,并收到 TorchScript 错误,因为参数的输入类型 yint 、 与默认参数不匹配类型, torch.Tensor .

def f2(x, y):
    return x + y

inp1 = torch.randn(5, 5)
inp2 = 3

script_f2 = torch.jit.script(f2)
try:
    script_f2(inp1, inp2)
except:
    tb.print_exc()
Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py", line 347, in <module>
    script_f2(inp1, inp2)
RuntimeError: f2() Expected a value of type 'Tensor (inferred)' for argument 'y' but instead found type 'int'.
Inferred 'y' to be of type 'Tensor' because it was not annotated with an explicit type.
Position: 1
Value: 3
Declaration: f2(Tensor x, Tensor y) -> Tensor
Cast error details: Unable to cast 3 to Tensor

然而, torch.compile 可以轻松处理 f2

compile_f2 = torch.compile(f2)
print("compile 2:", test_fns(f2, compile_f2, (inp1, inp2)))
print("~" * 10)
compile 2: True
~~~~~~~~~~

与之前的编译器解决方案相比, torch.compile 处理得很好的另一种情况是使用非 PyTorch 函数。

import scipy
def f3(x):
    x = x * 2
    x = scipy.fft.dct(x.numpy())
    x = torch.from_numpy(x)
    x = x * 2
    return x

TorchScript 跟踪将非 PyTorch 函数调用的结果视为常量,因此我们的结果可能会默默地错误。

inp1 = torch.randn(5, 5)
inp2 = torch.randn(5, 5)
traced_f3 = torch.jit.trace(f3, (inp1,))
print("traced 3:", test_fns(f3, traced_f3, (inp2,)))
/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py:365: TracerWarning:

Converting a tensor to a NumPy array might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!

/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py:366: TracerWarning:

torch.from_numpy results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.

traced 3: False

TorchScript 脚本和 FX 跟踪不允许非 PyTorch 函数调用。

try:
    torch.jit.script(f3)
except:
    tb.print_exc()

try:
    torch.fx.symbolic_trace(f3)
except:
    tb.print_exc()
Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py", line 383, in <module>
    torch.jit.script(f3)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/jit/_script.py", line 1381, in script
    fn = torch._C._jit_script_compile(
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_jit_internal.py", line 1205, in _try_get_dispatched_fn
    return boolean_dispatched.get(fn)
  File "/opt/conda/envs/py_3.10/lib/python3.10/weakref.py", line 453, in get
    return self.data.get(ref(key),default)
TypeError: cannot create weak reference to 'uarray._Function' object
Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py", line 388, in <module>
    torch.fx.symbolic_trace(f3)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 1150, in symbolic_trace
    graph = tracer.trace(root, concrete_args)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 328, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 17, in inner
    return fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 817, in trace
    (self.create_arg(fn(*args)),),
  File "/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py", line 365, in f3
    x = scipy.fft.dct(x.numpy())
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/scipy/fft/_backend.py", line 25, in __ua_function__
    return fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/scipy/fft/_pocketfft/realtransforms.py", line 19, in _r2r
    tmp = _asfarray(x)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/scipy/fft/_pocketfft/helper.py", line 89, in _asfarray
    if x.dtype == np.float16:
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/proxy.py", line 542, in impl
    return tracer.create_proxy('call_function', target, args, kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/proxy.py", line 184, in create_proxy
    args_ = self.create_arg(args)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 385, in create_arg
    return super().create_arg(a)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/proxy.py", line 255, in create_arg
    return type(a)(self.create_arg(elem) for elem in a)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/proxy.py", line 255, in <genexpr>
    return type(a)(self.create_arg(elem) for elem in a)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 385, in create_arg
    return super().create_arg(a)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/proxy.py", line 291, in create_arg
    raise NotImplementedError(f"argument of type: {type(a)}")
NotImplementedError: argument of type: <class 'type'>

相比之下, torch.compile 能够轻松处理 非 PyTorch 函数调用。

compile_f3 = torch.compile(f3)
print("compile 3:", test_fns(f3, compile_f3, (inp2,)))
compile 3: True

TorchDynamo 和 FX 图表

torch.compile 的一个重要组件是 TorchDynamo。 TorchDynamo 负责 JIT 将任意 Python 代码编译成 FX 图表 ,然后可以进一步优化。 TorchDynamo 通过在运行时分析 Python 字节码并检测对 PyTorch 操作的调用来提取 FX 图表。

通常,TorchInductor(torch.compile 的另一个组件) 进一步将 FX 图表编译为优化内核, 但 TorchDynamo 允许使用不同的后端。为了检查 TorchDynamo 输出的 FX 图表,让我们创建一个自定义后端, 输出 FX 图表并简单地返回图表’s 未优化的前向方法。

from typing import List
def custom_backend(gm: torch.fx.GraphModule, example_inputs: List[torch.Tensor]):
    print("custom backend called with FX graph:")
    gm.graph.print_tabular()
    return gm.forward

# Reset since we are using a different backend.
torch._dynamo.reset()

opt_model = torch.compile(init_model(), backend=custom_backend)
opt_model(generate_data(16)[0])
custom backend called with FX graph:
opcode         name                                               target                                                      args                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       kwargs
-------------  -------------------------------------------------  ----------------------------------------------------------  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------  -----------------
placeholder    l_x_                                               L_x_                                                        ()                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         {}
call_module    l__self___features_conv0                           L__self___features_conv0                                    (l_x_,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    {}
call_module    l__self___features_norm0                           L__self___features_norm0                                    (l__self___features_conv0,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                {}
call_module    l__self___features_relu0                           L__self___features_relu0                                    (l__self___features_norm0,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                {}
call_module    l__self___features_pool0                           L__self___features_pool0                                    (l__self___features_relu0,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                {}
call_function  cat                                                <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_pool0], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            {}
call_module    l__self___features_denseblock1_denselayer1_norm1   L__self___features_denseblock1_denselayer1_norm1            (cat,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_denseblock1_denselayer1_relu1   L__self___features_denseblock1_denselayer1_relu1            (l__self___features_denseblock1_denselayer1_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer1_conv1   L__self___features_denseblock1_denselayer1_conv1            (l__self___features_denseblock1_denselayer1_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer1_norm2   L__self___features_denseblock1_denselayer1_norm2            (l__self___features_denseblock1_denselayer1_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer1_relu2   L__self___features_denseblock1_denselayer1_relu2            (l__self___features_denseblock1_denselayer1_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer1_conv2   L__self___features_denseblock1_denselayer1_conv2            (l__self___features_denseblock1_denselayer1_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_1                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_pool0, l__self___features_denseblock1_denselayer1_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          {}
call_module    l__self___features_denseblock1_denselayer2_norm1   L__self___features_denseblock1_denselayer2_norm1            (cat_1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock1_denselayer2_relu1   L__self___features_denseblock1_denselayer2_relu1            (l__self___features_denseblock1_denselayer2_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer2_conv1   L__self___features_denseblock1_denselayer2_conv1            (l__self___features_denseblock1_denselayer2_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer2_norm2   L__self___features_denseblock1_denselayer2_norm2            (l__self___features_denseblock1_denselayer2_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer2_relu2   L__self___features_denseblock1_denselayer2_relu2            (l__self___features_denseblock1_denselayer2_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer2_conv2   L__self___features_denseblock1_denselayer2_conv2            (l__self___features_denseblock1_denselayer2_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_2                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_pool0, l__self___features_denseblock1_denselayer1_conv2, l__self___features_denseblock1_denselayer2_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer3_norm1   L__self___features_denseblock1_denselayer3_norm1            (cat_2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock1_denselayer3_relu1   L__self___features_denseblock1_denselayer3_relu1            (l__self___features_denseblock1_denselayer3_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer3_conv1   L__self___features_denseblock1_denselayer3_conv1            (l__self___features_denseblock1_denselayer3_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer3_norm2   L__self___features_denseblock1_denselayer3_norm2            (l__self___features_denseblock1_denselayer3_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer3_relu2   L__self___features_denseblock1_denselayer3_relu2            (l__self___features_denseblock1_denselayer3_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer3_conv2   L__self___features_denseblock1_denselayer3_conv2            (l__self___features_denseblock1_denselayer3_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_3                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_pool0, l__self___features_denseblock1_denselayer1_conv2, l__self___features_denseblock1_denselayer2_conv2, l__self___features_denseblock1_denselayer3_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      {}
call_module    l__self___features_denseblock1_denselayer4_norm1   L__self___features_denseblock1_denselayer4_norm1            (cat_3,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock1_denselayer4_relu1   L__self___features_denseblock1_denselayer4_relu1            (l__self___features_denseblock1_denselayer4_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer4_conv1   L__self___features_denseblock1_denselayer4_conv1            (l__self___features_denseblock1_denselayer4_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer4_norm2   L__self___features_denseblock1_denselayer4_norm2            (l__self___features_denseblock1_denselayer4_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer4_relu2   L__self___features_denseblock1_denselayer4_relu2            (l__self___features_denseblock1_denselayer4_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer4_conv2   L__self___features_denseblock1_denselayer4_conv2            (l__self___features_denseblock1_denselayer4_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_4                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_pool0, l__self___features_denseblock1_denselayer1_conv2, l__self___features_denseblock1_denselayer2_conv2, l__self___features_denseblock1_denselayer3_conv2, l__self___features_denseblock1_denselayer4_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    {}
call_module    l__self___features_denseblock1_denselayer5_norm1   L__self___features_denseblock1_denselayer5_norm1            (cat_4,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock1_denselayer5_relu1   L__self___features_denseblock1_denselayer5_relu1            (l__self___features_denseblock1_denselayer5_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer5_conv1   L__self___features_denseblock1_denselayer5_conv1            (l__self___features_denseblock1_denselayer5_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer5_norm2   L__self___features_denseblock1_denselayer5_norm2            (l__self___features_denseblock1_denselayer5_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer5_relu2   L__self___features_denseblock1_denselayer5_relu2            (l__self___features_denseblock1_denselayer5_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer5_conv2   L__self___features_denseblock1_denselayer5_conv2            (l__self___features_denseblock1_denselayer5_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_5                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_pool0, l__self___features_denseblock1_denselayer1_conv2, l__self___features_denseblock1_denselayer2_conv2, l__self___features_denseblock1_denselayer3_conv2, l__self___features_denseblock1_denselayer4_conv2, l__self___features_denseblock1_denselayer5_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock1_denselayer6_norm1   L__self___features_denseblock1_denselayer6_norm1            (cat_5,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock1_denselayer6_relu1   L__self___features_denseblock1_denselayer6_relu1            (l__self___features_denseblock1_denselayer6_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer6_conv1   L__self___features_denseblock1_denselayer6_conv1            (l__self___features_denseblock1_denselayer6_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer6_norm2   L__self___features_denseblock1_denselayer6_norm2            (l__self___features_denseblock1_denselayer6_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer6_relu2   L__self___features_denseblock1_denselayer6_relu2            (l__self___features_denseblock1_denselayer6_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock1_denselayer6_conv2   L__self___features_denseblock1_denselayer6_conv2            (l__self___features_denseblock1_denselayer6_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_6                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_pool0, l__self___features_denseblock1_denselayer1_conv2, l__self___features_denseblock1_denselayer2_conv2, l__self___features_denseblock1_denselayer3_conv2, l__self___features_denseblock1_denselayer4_conv2, l__self___features_denseblock1_denselayer5_conv2, l__self___features_denseblock1_denselayer6_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                {}
call_module    l__self___features_transition1_norm                L__self___features_transition1_norm                         (cat_6,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_transition1_relu                L__self___features_transition1_relu                         (l__self___features_transition1_norm,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_transition1_conv                L__self___features_transition1_conv                         (l__self___features_transition1_relu,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_transition1_pool                L__self___features_transition1_pool                         (l__self___features_transition1_conv,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_function  cat_7                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 {}
call_module    l__self___features_denseblock2_denselayer1_norm1   L__self___features_denseblock2_denselayer1_norm1            (cat_7,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock2_denselayer1_relu1   L__self___features_denseblock2_denselayer1_relu1            (l__self___features_denseblock2_denselayer1_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer1_conv1   L__self___features_denseblock2_denselayer1_conv1            (l__self___features_denseblock2_denselayer1_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer1_norm2   L__self___features_denseblock2_denselayer1_norm2            (l__self___features_denseblock2_denselayer1_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer1_relu2   L__self___features_denseblock2_denselayer1_relu2            (l__self___features_denseblock2_denselayer1_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer1_conv2   L__self___features_denseblock2_denselayer1_conv2            (l__self___features_denseblock2_denselayer1_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_8                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               {}
call_module    l__self___features_denseblock2_denselayer2_norm1   L__self___features_denseblock2_denselayer2_norm1            (cat_8,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock2_denselayer2_relu1   L__self___features_denseblock2_denselayer2_relu1            (l__self___features_denseblock2_denselayer2_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer2_conv1   L__self___features_denseblock2_denselayer2_conv1            (l__self___features_denseblock2_denselayer2_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer2_norm2   L__self___features_denseblock2_denselayer2_norm2            (l__self___features_denseblock2_denselayer2_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer2_relu2   L__self___features_denseblock2_denselayer2_relu2            (l__self___features_denseblock2_denselayer2_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer2_conv2   L__self___features_denseblock2_denselayer2_conv2            (l__self___features_denseblock2_denselayer2_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_9                                              <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             {}
call_module    l__self___features_denseblock2_denselayer3_norm1   L__self___features_denseblock2_denselayer3_norm1            (cat_9,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock2_denselayer3_relu1   L__self___features_denseblock2_denselayer3_relu1            (l__self___features_denseblock2_denselayer3_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer3_conv1   L__self___features_denseblock2_denselayer3_conv1            (l__self___features_denseblock2_denselayer3_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer3_norm2   L__self___features_denseblock2_denselayer3_norm2            (l__self___features_denseblock2_denselayer3_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer3_relu2   L__self___features_denseblock2_denselayer3_relu2            (l__self___features_denseblock2_denselayer3_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer3_conv2   L__self___features_denseblock2_denselayer3_conv2            (l__self___features_denseblock2_denselayer3_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_10                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           {}
call_module    l__self___features_denseblock2_denselayer4_norm1   L__self___features_denseblock2_denselayer4_norm1            (cat_10,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer4_relu1   L__self___features_denseblock2_denselayer4_relu1            (l__self___features_denseblock2_denselayer4_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer4_conv1   L__self___features_denseblock2_denselayer4_conv1            (l__self___features_denseblock2_denselayer4_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer4_norm2   L__self___features_denseblock2_denselayer4_norm2            (l__self___features_denseblock2_denselayer4_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer4_relu2   L__self___features_denseblock2_denselayer4_relu2            (l__self___features_denseblock2_denselayer4_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer4_conv2   L__self___features_denseblock2_denselayer4_conv2            (l__self___features_denseblock2_denselayer4_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_11                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         {}
call_module    l__self___features_denseblock2_denselayer5_norm1   L__self___features_denseblock2_denselayer5_norm1            (cat_11,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer5_relu1   L__self___features_denseblock2_denselayer5_relu1            (l__self___features_denseblock2_denselayer5_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer5_conv1   L__self___features_denseblock2_denselayer5_conv1            (l__self___features_denseblock2_denselayer5_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer5_norm2   L__self___features_denseblock2_denselayer5_norm2            (l__self___features_denseblock2_denselayer5_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer5_relu2   L__self___features_denseblock2_denselayer5_relu2            (l__self___features_denseblock2_denselayer5_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer5_conv2   L__self___features_denseblock2_denselayer5_conv2            (l__self___features_denseblock2_denselayer5_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_12                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2, l__self___features_denseblock2_denselayer5_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer6_norm1   L__self___features_denseblock2_denselayer6_norm1            (cat_12,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer6_relu1   L__self___features_denseblock2_denselayer6_relu1            (l__self___features_denseblock2_denselayer6_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer6_conv1   L__self___features_denseblock2_denselayer6_conv1            (l__self___features_denseblock2_denselayer6_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer6_norm2   L__self___features_denseblock2_denselayer6_norm2            (l__self___features_denseblock2_denselayer6_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer6_relu2   L__self___features_denseblock2_denselayer6_relu2            (l__self___features_denseblock2_denselayer6_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer6_conv2   L__self___features_denseblock2_denselayer6_conv2            (l__self___features_denseblock2_denselayer6_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_13                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2, l__self___features_denseblock2_denselayer5_conv2, l__self___features_denseblock2_denselayer6_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_denseblock2_denselayer7_norm1   L__self___features_denseblock2_denselayer7_norm1            (cat_13,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer7_relu1   L__self___features_denseblock2_denselayer7_relu1            (l__self___features_denseblock2_denselayer7_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer7_conv1   L__self___features_denseblock2_denselayer7_conv1            (l__self___features_denseblock2_denselayer7_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer7_norm2   L__self___features_denseblock2_denselayer7_norm2            (l__self___features_denseblock2_denselayer7_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer7_relu2   L__self___features_denseblock2_denselayer7_relu2            (l__self___features_denseblock2_denselayer7_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer7_conv2   L__self___features_denseblock2_denselayer7_conv2            (l__self___features_denseblock2_denselayer7_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_14                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2, l__self___features_denseblock2_denselayer5_conv2, l__self___features_denseblock2_denselayer6_conv2, l__self___features_denseblock2_denselayer7_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock2_denselayer8_norm1   L__self___features_denseblock2_denselayer8_norm1            (cat_14,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer8_relu1   L__self___features_denseblock2_denselayer8_relu1            (l__self___features_denseblock2_denselayer8_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer8_conv1   L__self___features_denseblock2_denselayer8_conv1            (l__self___features_denseblock2_denselayer8_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer8_norm2   L__self___features_denseblock2_denselayer8_norm2            (l__self___features_denseblock2_denselayer8_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer8_relu2   L__self___features_denseblock2_denselayer8_relu2            (l__self___features_denseblock2_denselayer8_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer8_conv2   L__self___features_denseblock2_denselayer8_conv2            (l__self___features_denseblock2_denselayer8_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_15                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2, l__self___features_denseblock2_denselayer5_conv2, l__self___features_denseblock2_denselayer6_conv2, l__self___features_denseblock2_denselayer7_conv2, l__self___features_denseblock2_denselayer8_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 {}
call_module    l__self___features_denseblock2_denselayer9_norm1   L__self___features_denseblock2_denselayer9_norm1            (cat_15,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer9_relu1   L__self___features_denseblock2_denselayer9_relu1            (l__self___features_denseblock2_denselayer9_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer9_conv1   L__self___features_denseblock2_denselayer9_conv1            (l__self___features_denseblock2_denselayer9_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer9_norm2   L__self___features_denseblock2_denselayer9_norm2            (l__self___features_denseblock2_denselayer9_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer9_relu2   L__self___features_denseblock2_denselayer9_relu2            (l__self___features_denseblock2_denselayer9_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock2_denselayer9_conv2   L__self___features_denseblock2_denselayer9_conv2            (l__self___features_denseblock2_denselayer9_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_16                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2, l__self___features_denseblock2_denselayer5_conv2, l__self___features_denseblock2_denselayer6_conv2, l__self___features_denseblock2_denselayer7_conv2, l__self___features_denseblock2_denselayer8_conv2, l__self___features_denseblock2_denselayer9_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               {}
call_module    l__self___features_denseblock2_denselayer10_norm1  L__self___features_denseblock2_denselayer10_norm1           (cat_16,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer10_relu1  L__self___features_denseblock2_denselayer10_relu1           (l__self___features_denseblock2_denselayer10_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer10_conv1  L__self___features_denseblock2_denselayer10_conv1           (l__self___features_denseblock2_denselayer10_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer10_norm2  L__self___features_denseblock2_denselayer10_norm2           (l__self___features_denseblock2_denselayer10_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer10_relu2  L__self___features_denseblock2_denselayer10_relu2           (l__self___features_denseblock2_denselayer10_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer10_conv2  L__self___features_denseblock2_denselayer10_conv2           (l__self___features_denseblock2_denselayer10_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_17                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2, l__self___features_denseblock2_denselayer5_conv2, l__self___features_denseblock2_denselayer6_conv2, l__self___features_denseblock2_denselayer7_conv2, l__self___features_denseblock2_denselayer8_conv2, l__self___features_denseblock2_denselayer9_conv2, l__self___features_denseblock2_denselayer10_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            {}
call_module    l__self___features_denseblock2_denselayer11_norm1  L__self___features_denseblock2_denselayer11_norm1           (cat_17,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer11_relu1  L__self___features_denseblock2_denselayer11_relu1           (l__self___features_denseblock2_denselayer11_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer11_conv1  L__self___features_denseblock2_denselayer11_conv1           (l__self___features_denseblock2_denselayer11_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer11_norm2  L__self___features_denseblock2_denselayer11_norm2           (l__self___features_denseblock2_denselayer11_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer11_relu2  L__self___features_denseblock2_denselayer11_relu2           (l__self___features_denseblock2_denselayer11_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer11_conv2  L__self___features_denseblock2_denselayer11_conv2           (l__self___features_denseblock2_denselayer11_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_18                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2, l__self___features_denseblock2_denselayer5_conv2, l__self___features_denseblock2_denselayer6_conv2, l__self___features_denseblock2_denselayer7_conv2, l__self___features_denseblock2_denselayer8_conv2, l__self___features_denseblock2_denselayer9_conv2, l__self___features_denseblock2_denselayer10_conv2, l__self___features_denseblock2_denselayer11_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         {}
call_module    l__self___features_denseblock2_denselayer12_norm1  L__self___features_denseblock2_denselayer12_norm1           (cat_18,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock2_denselayer12_relu1  L__self___features_denseblock2_denselayer12_relu1           (l__self___features_denseblock2_denselayer12_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer12_conv1  L__self___features_denseblock2_denselayer12_conv1           (l__self___features_denseblock2_denselayer12_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer12_norm2  L__self___features_denseblock2_denselayer12_norm2           (l__self___features_denseblock2_denselayer12_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer12_relu2  L__self___features_denseblock2_denselayer12_relu2           (l__self___features_denseblock2_denselayer12_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock2_denselayer12_conv2  L__self___features_denseblock2_denselayer12_conv2           (l__self___features_denseblock2_denselayer12_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_19                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition1_pool, l__self___features_denseblock2_denselayer1_conv2, l__self___features_denseblock2_denselayer2_conv2, l__self___features_denseblock2_denselayer3_conv2, l__self___features_denseblock2_denselayer4_conv2, l__self___features_denseblock2_denselayer5_conv2, l__self___features_denseblock2_denselayer6_conv2, l__self___features_denseblock2_denselayer7_conv2, l__self___features_denseblock2_denselayer8_conv2, l__self___features_denseblock2_denselayer9_conv2, l__self___features_denseblock2_denselayer10_conv2, l__self___features_denseblock2_denselayer11_conv2, l__self___features_denseblock2_denselayer12_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      {}
call_module    l__self___features_transition2_norm                L__self___features_transition2_norm                         (cat_19,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_transition2_relu                L__self___features_transition2_relu                         (l__self___features_transition2_norm,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_transition2_conv                L__self___features_transition2_conv                         (l__self___features_transition2_relu,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_transition2_pool                L__self___features_transition2_pool                         (l__self___features_transition2_conv,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_function  cat_20                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 {}
call_module    l__self___features_denseblock3_denselayer1_norm1   L__self___features_denseblock3_denselayer1_norm1            (cat_20,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer1_relu1   L__self___features_denseblock3_denselayer1_relu1            (l__self___features_denseblock3_denselayer1_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer1_conv1   L__self___features_denseblock3_denselayer1_conv1            (l__self___features_denseblock3_denselayer1_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer1_norm2   L__self___features_denseblock3_denselayer1_norm2            (l__self___features_denseblock3_denselayer1_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer1_relu2   L__self___features_denseblock3_denselayer1_relu2            (l__self___features_denseblock3_denselayer1_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer1_conv2   L__self___features_denseblock3_denselayer1_conv2            (l__self___features_denseblock3_denselayer1_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_21                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               {}
call_module    l__self___features_denseblock3_denselayer2_norm1   L__self___features_denseblock3_denselayer2_norm1            (cat_21,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer2_relu1   L__self___features_denseblock3_denselayer2_relu1            (l__self___features_denseblock3_denselayer2_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer2_conv1   L__self___features_denseblock3_denselayer2_conv1            (l__self___features_denseblock3_denselayer2_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer2_norm2   L__self___features_denseblock3_denselayer2_norm2            (l__self___features_denseblock3_denselayer2_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer2_relu2   L__self___features_denseblock3_denselayer2_relu2            (l__self___features_denseblock3_denselayer2_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer2_conv2   L__self___features_denseblock3_denselayer2_conv2            (l__self___features_denseblock3_denselayer2_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_22                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             {}
call_module    l__self___features_denseblock3_denselayer3_norm1   L__self___features_denseblock3_denselayer3_norm1            (cat_22,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer3_relu1   L__self___features_denseblock3_denselayer3_relu1            (l__self___features_denseblock3_denselayer3_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer3_conv1   L__self___features_denseblock3_denselayer3_conv1            (l__self___features_denseblock3_denselayer3_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer3_norm2   L__self___features_denseblock3_denselayer3_norm2            (l__self___features_denseblock3_denselayer3_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer3_relu2   L__self___features_denseblock3_denselayer3_relu2            (l__self___features_denseblock3_denselayer3_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer3_conv2   L__self___features_denseblock3_denselayer3_conv2            (l__self___features_denseblock3_denselayer3_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_23                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           {}
call_module    l__self___features_denseblock3_denselayer4_norm1   L__self___features_denseblock3_denselayer4_norm1            (cat_23,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer4_relu1   L__self___features_denseblock3_denselayer4_relu1            (l__self___features_denseblock3_denselayer4_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer4_conv1   L__self___features_denseblock3_denselayer4_conv1            (l__self___features_denseblock3_denselayer4_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer4_norm2   L__self___features_denseblock3_denselayer4_norm2            (l__self___features_denseblock3_denselayer4_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer4_relu2   L__self___features_denseblock3_denselayer4_relu2            (l__self___features_denseblock3_denselayer4_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer4_conv2   L__self___features_denseblock3_denselayer4_conv2            (l__self___features_denseblock3_denselayer4_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_24                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         {}
call_module    l__self___features_denseblock3_denselayer5_norm1   L__self___features_denseblock3_denselayer5_norm1            (cat_24,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer5_relu1   L__self___features_denseblock3_denselayer5_relu1            (l__self___features_denseblock3_denselayer5_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer5_conv1   L__self___features_denseblock3_denselayer5_conv1            (l__self___features_denseblock3_denselayer5_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer5_norm2   L__self___features_denseblock3_denselayer5_norm2            (l__self___features_denseblock3_denselayer5_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer5_relu2   L__self___features_denseblock3_denselayer5_relu2            (l__self___features_denseblock3_denselayer5_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer5_conv2   L__self___features_denseblock3_denselayer5_conv2            (l__self___features_denseblock3_denselayer5_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_25                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer6_norm1   L__self___features_denseblock3_denselayer6_norm1            (cat_25,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer6_relu1   L__self___features_denseblock3_denselayer6_relu1            (l__self___features_denseblock3_denselayer6_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer6_conv1   L__self___features_denseblock3_denselayer6_conv1            (l__self___features_denseblock3_denselayer6_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer6_norm2   L__self___features_denseblock3_denselayer6_norm2            (l__self___features_denseblock3_denselayer6_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer6_relu2   L__self___features_denseblock3_denselayer6_relu2            (l__self___features_denseblock3_denselayer6_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer6_conv2   L__self___features_denseblock3_denselayer6_conv2            (l__self___features_denseblock3_denselayer6_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_26                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_denseblock3_denselayer7_norm1   L__self___features_denseblock3_denselayer7_norm1            (cat_26,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer7_relu1   L__self___features_denseblock3_denselayer7_relu1            (l__self___features_denseblock3_denselayer7_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer7_conv1   L__self___features_denseblock3_denselayer7_conv1            (l__self___features_denseblock3_denselayer7_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer7_norm2   L__self___features_denseblock3_denselayer7_norm2            (l__self___features_denseblock3_denselayer7_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer7_relu2   L__self___features_denseblock3_denselayer7_relu2            (l__self___features_denseblock3_denselayer7_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer7_conv2   L__self___features_denseblock3_denselayer7_conv2            (l__self___features_denseblock3_denselayer7_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_27                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock3_denselayer8_norm1   L__self___features_denseblock3_denselayer8_norm1            (cat_27,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer8_relu1   L__self___features_denseblock3_denselayer8_relu1            (l__self___features_denseblock3_denselayer8_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer8_conv1   L__self___features_denseblock3_denselayer8_conv1            (l__self___features_denseblock3_denselayer8_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer8_norm2   L__self___features_denseblock3_denselayer8_norm2            (l__self___features_denseblock3_denselayer8_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer8_relu2   L__self___features_denseblock3_denselayer8_relu2            (l__self___features_denseblock3_denselayer8_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer8_conv2   L__self___features_denseblock3_denselayer8_conv2            (l__self___features_denseblock3_denselayer8_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_28                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 {}
call_module    l__self___features_denseblock3_denselayer9_norm1   L__self___features_denseblock3_denselayer9_norm1            (cat_28,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer9_relu1   L__self___features_denseblock3_denselayer9_relu1            (l__self___features_denseblock3_denselayer9_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer9_conv1   L__self___features_denseblock3_denselayer9_conv1            (l__self___features_denseblock3_denselayer9_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer9_norm2   L__self___features_denseblock3_denselayer9_norm2            (l__self___features_denseblock3_denselayer9_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer9_relu2   L__self___features_denseblock3_denselayer9_relu2            (l__self___features_denseblock3_denselayer9_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer9_conv2   L__self___features_denseblock3_denselayer9_conv2            (l__self___features_denseblock3_denselayer9_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_29                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               {}
call_module    l__self___features_denseblock3_denselayer10_norm1  L__self___features_denseblock3_denselayer10_norm1           (cat_29,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer10_relu1  L__self___features_denseblock3_denselayer10_relu1           (l__self___features_denseblock3_denselayer10_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer10_conv1  L__self___features_denseblock3_denselayer10_conv1           (l__self___features_denseblock3_denselayer10_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer10_norm2  L__self___features_denseblock3_denselayer10_norm2           (l__self___features_denseblock3_denselayer10_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer10_relu2  L__self___features_denseblock3_denselayer10_relu2           (l__self___features_denseblock3_denselayer10_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer10_conv2  L__self___features_denseblock3_denselayer10_conv2           (l__self___features_denseblock3_denselayer10_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_30                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            {}
call_module    l__self___features_denseblock3_denselayer11_norm1  L__self___features_denseblock3_denselayer11_norm1           (cat_30,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer11_relu1  L__self___features_denseblock3_denselayer11_relu1           (l__self___features_denseblock3_denselayer11_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer11_conv1  L__self___features_denseblock3_denselayer11_conv1           (l__self___features_denseblock3_denselayer11_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer11_norm2  L__self___features_denseblock3_denselayer11_norm2           (l__self___features_denseblock3_denselayer11_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer11_relu2  L__self___features_denseblock3_denselayer11_relu2           (l__self___features_denseblock3_denselayer11_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer11_conv2  L__self___features_denseblock3_denselayer11_conv2           (l__self___features_denseblock3_denselayer11_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_31                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         {}
call_module    l__self___features_denseblock3_denselayer12_norm1  L__self___features_denseblock3_denselayer12_norm1           (cat_31,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer12_relu1  L__self___features_denseblock3_denselayer12_relu1           (l__self___features_denseblock3_denselayer12_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer12_conv1  L__self___features_denseblock3_denselayer12_conv1           (l__self___features_denseblock3_denselayer12_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer12_norm2  L__self___features_denseblock3_denselayer12_norm2           (l__self___features_denseblock3_denselayer12_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer12_relu2  L__self___features_denseblock3_denselayer12_relu2           (l__self___features_denseblock3_denselayer12_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer12_conv2  L__self___features_denseblock3_denselayer12_conv2           (l__self___features_denseblock3_denselayer12_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_32                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      {}
call_module    l__self___features_denseblock3_denselayer13_norm1  L__self___features_denseblock3_denselayer13_norm1           (cat_32,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer13_relu1  L__self___features_denseblock3_denselayer13_relu1           (l__self___features_denseblock3_denselayer13_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer13_conv1  L__self___features_denseblock3_denselayer13_conv1           (l__self___features_denseblock3_denselayer13_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer13_norm2  L__self___features_denseblock3_denselayer13_norm2           (l__self___features_denseblock3_denselayer13_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer13_relu2  L__self___features_denseblock3_denselayer13_relu2           (l__self___features_denseblock3_denselayer13_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer13_conv2  L__self___features_denseblock3_denselayer13_conv2           (l__self___features_denseblock3_denselayer13_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_33                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock3_denselayer14_norm1  L__self___features_denseblock3_denselayer14_norm1           (cat_33,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer14_relu1  L__self___features_denseblock3_denselayer14_relu1           (l__self___features_denseblock3_denselayer14_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer14_conv1  L__self___features_denseblock3_denselayer14_conv1           (l__self___features_denseblock3_denselayer14_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer14_norm2  L__self___features_denseblock3_denselayer14_norm2           (l__self___features_denseblock3_denselayer14_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer14_relu2  L__self___features_denseblock3_denselayer14_relu2           (l__self___features_denseblock3_denselayer14_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer14_conv2  L__self___features_denseblock3_denselayer14_conv2           (l__self___features_denseblock3_denselayer14_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_34                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                {}
call_module    l__self___features_denseblock3_denselayer15_norm1  L__self___features_denseblock3_denselayer15_norm1           (cat_34,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer15_relu1  L__self___features_denseblock3_denselayer15_relu1           (l__self___features_denseblock3_denselayer15_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer15_conv1  L__self___features_denseblock3_denselayer15_conv1           (l__self___features_denseblock3_denselayer15_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer15_norm2  L__self___features_denseblock3_denselayer15_norm2           (l__self___features_denseblock3_denselayer15_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer15_relu2  L__self___features_denseblock3_denselayer15_relu2           (l__self___features_denseblock3_denselayer15_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer15_conv2  L__self___features_denseblock3_denselayer15_conv2           (l__self___features_denseblock3_denselayer15_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_35                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                             {}
call_module    l__self___features_denseblock3_denselayer16_norm1  L__self___features_denseblock3_denselayer16_norm1           (cat_35,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer16_relu1  L__self___features_denseblock3_denselayer16_relu1           (l__self___features_denseblock3_denselayer16_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer16_conv1  L__self___features_denseblock3_denselayer16_conv1           (l__self___features_denseblock3_denselayer16_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer16_norm2  L__self___features_denseblock3_denselayer16_norm2           (l__self___features_denseblock3_denselayer16_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer16_relu2  L__self___features_denseblock3_denselayer16_relu2           (l__self___features_denseblock3_denselayer16_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer16_conv2  L__self___features_denseblock3_denselayer16_conv2           (l__self___features_denseblock3_denselayer16_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_36                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                          {}
call_module    l__self___features_denseblock3_denselayer17_norm1  L__self___features_denseblock3_denselayer17_norm1           (cat_36,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer17_relu1  L__self___features_denseblock3_denselayer17_relu1           (l__self___features_denseblock3_denselayer17_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer17_conv1  L__self___features_denseblock3_denselayer17_conv1           (l__self___features_denseblock3_denselayer17_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer17_norm2  L__self___features_denseblock3_denselayer17_norm2           (l__self___features_denseblock3_denselayer17_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer17_relu2  L__self___features_denseblock3_denselayer17_relu2           (l__self___features_denseblock3_denselayer17_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer17_conv2  L__self___features_denseblock3_denselayer17_conv2           (l__self___features_denseblock3_denselayer17_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_37                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2, l__self___features_denseblock3_denselayer17_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer18_norm1  L__self___features_denseblock3_denselayer18_norm1           (cat_37,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer18_relu1  L__self___features_denseblock3_denselayer18_relu1           (l__self___features_denseblock3_denselayer18_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer18_conv1  L__self___features_denseblock3_denselayer18_conv1           (l__self___features_denseblock3_denselayer18_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer18_norm2  L__self___features_denseblock3_denselayer18_norm2           (l__self___features_denseblock3_denselayer18_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer18_relu2  L__self___features_denseblock3_denselayer18_relu2           (l__self___features_denseblock3_denselayer18_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer18_conv2  L__self___features_denseblock3_denselayer18_conv2           (l__self___features_denseblock3_denselayer18_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_38                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2, l__self___features_denseblock3_denselayer17_conv2, l__self___features_denseblock3_denselayer18_conv2], 1)                                                                                                                                                                                                                                                                                                                    {}
call_module    l__self___features_denseblock3_denselayer19_norm1  L__self___features_denseblock3_denselayer19_norm1           (cat_38,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer19_relu1  L__self___features_denseblock3_denselayer19_relu1           (l__self___features_denseblock3_denselayer19_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer19_conv1  L__self___features_denseblock3_denselayer19_conv1           (l__self___features_denseblock3_denselayer19_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer19_norm2  L__self___features_denseblock3_denselayer19_norm2           (l__self___features_denseblock3_denselayer19_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer19_relu2  L__self___features_denseblock3_denselayer19_relu2           (l__self___features_denseblock3_denselayer19_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer19_conv2  L__self___features_denseblock3_denselayer19_conv2           (l__self___features_denseblock3_denselayer19_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_39                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2, l__self___features_denseblock3_denselayer17_conv2, l__self___features_denseblock3_denselayer18_conv2, l__self___features_denseblock3_denselayer19_conv2], 1)                                                                                                                                                                                                                                                                 {}
call_module    l__self___features_denseblock3_denselayer20_norm1  L__self___features_denseblock3_denselayer20_norm1           (cat_39,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer20_relu1  L__self___features_denseblock3_denselayer20_relu1           (l__self___features_denseblock3_denselayer20_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer20_conv1  L__self___features_denseblock3_denselayer20_conv1           (l__self___features_denseblock3_denselayer20_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer20_norm2  L__self___features_denseblock3_denselayer20_norm2           (l__self___features_denseblock3_denselayer20_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer20_relu2  L__self___features_denseblock3_denselayer20_relu2           (l__self___features_denseblock3_denselayer20_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer20_conv2  L__self___features_denseblock3_denselayer20_conv2           (l__self___features_denseblock3_denselayer20_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_40                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2, l__self___features_denseblock3_denselayer17_conv2, l__self___features_denseblock3_denselayer18_conv2, l__self___features_denseblock3_denselayer19_conv2, l__self___features_denseblock3_denselayer20_conv2], 1)                                                                                                                                                                                                              {}
call_module    l__self___features_denseblock3_denselayer21_norm1  L__self___features_denseblock3_denselayer21_norm1           (cat_40,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer21_relu1  L__self___features_denseblock3_denselayer21_relu1           (l__self___features_denseblock3_denselayer21_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer21_conv1  L__self___features_denseblock3_denselayer21_conv1           (l__self___features_denseblock3_denselayer21_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer21_norm2  L__self___features_denseblock3_denselayer21_norm2           (l__self___features_denseblock3_denselayer21_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer21_relu2  L__self___features_denseblock3_denselayer21_relu2           (l__self___features_denseblock3_denselayer21_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer21_conv2  L__self___features_denseblock3_denselayer21_conv2           (l__self___features_denseblock3_denselayer21_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_41                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2, l__self___features_denseblock3_denselayer17_conv2, l__self___features_denseblock3_denselayer18_conv2, l__self___features_denseblock3_denselayer19_conv2, l__self___features_denseblock3_denselayer20_conv2, l__self___features_denseblock3_denselayer21_conv2], 1)                                                                                                                                                           {}
call_module    l__self___features_denseblock3_denselayer22_norm1  L__self___features_denseblock3_denselayer22_norm1           (cat_41,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer22_relu1  L__self___features_denseblock3_denselayer22_relu1           (l__self___features_denseblock3_denselayer22_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer22_conv1  L__self___features_denseblock3_denselayer22_conv1           (l__self___features_denseblock3_denselayer22_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer22_norm2  L__self___features_denseblock3_denselayer22_norm2           (l__self___features_denseblock3_denselayer22_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer22_relu2  L__self___features_denseblock3_denselayer22_relu2           (l__self___features_denseblock3_denselayer22_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer22_conv2  L__self___features_denseblock3_denselayer22_conv2           (l__self___features_denseblock3_denselayer22_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_42                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2, l__self___features_denseblock3_denselayer17_conv2, l__self___features_denseblock3_denselayer18_conv2, l__self___features_denseblock3_denselayer19_conv2, l__self___features_denseblock3_denselayer20_conv2, l__self___features_denseblock3_denselayer21_conv2, l__self___features_denseblock3_denselayer22_conv2], 1)                                                                                                        {}
call_module    l__self___features_denseblock3_denselayer23_norm1  L__self___features_denseblock3_denselayer23_norm1           (cat_42,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer23_relu1  L__self___features_denseblock3_denselayer23_relu1           (l__self___features_denseblock3_denselayer23_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer23_conv1  L__self___features_denseblock3_denselayer23_conv1           (l__self___features_denseblock3_denselayer23_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer23_norm2  L__self___features_denseblock3_denselayer23_norm2           (l__self___features_denseblock3_denselayer23_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer23_relu2  L__self___features_denseblock3_denselayer23_relu2           (l__self___features_denseblock3_denselayer23_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer23_conv2  L__self___features_denseblock3_denselayer23_conv2           (l__self___features_denseblock3_denselayer23_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_43                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2, l__self___features_denseblock3_denselayer17_conv2, l__self___features_denseblock3_denselayer18_conv2, l__self___features_denseblock3_denselayer19_conv2, l__self___features_denseblock3_denselayer20_conv2, l__self___features_denseblock3_denselayer21_conv2, l__self___features_denseblock3_denselayer22_conv2, l__self___features_denseblock3_denselayer23_conv2], 1)                                                     {}
call_module    l__self___features_denseblock3_denselayer24_norm1  L__self___features_denseblock3_denselayer24_norm1           (cat_43,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock3_denselayer24_relu1  L__self___features_denseblock3_denselayer24_relu1           (l__self___features_denseblock3_denselayer24_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer24_conv1  L__self___features_denseblock3_denselayer24_conv1           (l__self___features_denseblock3_denselayer24_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer24_norm2  L__self___features_denseblock3_denselayer24_norm2           (l__self___features_denseblock3_denselayer24_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer24_relu2  L__self___features_denseblock3_denselayer24_relu2           (l__self___features_denseblock3_denselayer24_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock3_denselayer24_conv2  L__self___features_denseblock3_denselayer24_conv2           (l__self___features_denseblock3_denselayer24_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_44                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition2_pool, l__self___features_denseblock3_denselayer1_conv2, l__self___features_denseblock3_denselayer2_conv2, l__self___features_denseblock3_denselayer3_conv2, l__self___features_denseblock3_denselayer4_conv2, l__self___features_denseblock3_denselayer5_conv2, l__self___features_denseblock3_denselayer6_conv2, l__self___features_denseblock3_denselayer7_conv2, l__self___features_denseblock3_denselayer8_conv2, l__self___features_denseblock3_denselayer9_conv2, l__self___features_denseblock3_denselayer10_conv2, l__self___features_denseblock3_denselayer11_conv2, l__self___features_denseblock3_denselayer12_conv2, l__self___features_denseblock3_denselayer13_conv2, l__self___features_denseblock3_denselayer14_conv2, l__self___features_denseblock3_denselayer15_conv2, l__self___features_denseblock3_denselayer16_conv2, l__self___features_denseblock3_denselayer17_conv2, l__self___features_denseblock3_denselayer18_conv2, l__self___features_denseblock3_denselayer19_conv2, l__self___features_denseblock3_denselayer20_conv2, l__self___features_denseblock3_denselayer21_conv2, l__self___features_denseblock3_denselayer22_conv2, l__self___features_denseblock3_denselayer23_conv2, l__self___features_denseblock3_denselayer24_conv2], 1)  {}
call_module    l__self___features_transition3_norm                L__self___features_transition3_norm                         (cat_44,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_transition3_relu                L__self___features_transition3_relu                         (l__self___features_transition3_norm,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_transition3_conv                L__self___features_transition3_conv                         (l__self___features_transition3_relu,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_transition3_pool                L__self___features_transition3_pool                         (l__self___features_transition3_conv,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_function  cat_45                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 {}
call_module    l__self___features_denseblock4_denselayer1_norm1   L__self___features_denseblock4_denselayer1_norm1            (cat_45,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer1_relu1   L__self___features_denseblock4_denselayer1_relu1            (l__self___features_denseblock4_denselayer1_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer1_conv1   L__self___features_denseblock4_denselayer1_conv1            (l__self___features_denseblock4_denselayer1_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer1_norm2   L__self___features_denseblock4_denselayer1_norm2            (l__self___features_denseblock4_denselayer1_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer1_relu2   L__self___features_denseblock4_denselayer1_relu2            (l__self___features_denseblock4_denselayer1_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer1_conv2   L__self___features_denseblock4_denselayer1_conv2            (l__self___features_denseblock4_denselayer1_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_46                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               {}
call_module    l__self___features_denseblock4_denselayer2_norm1   L__self___features_denseblock4_denselayer2_norm1            (cat_46,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer2_relu1   L__self___features_denseblock4_denselayer2_relu1            (l__self___features_denseblock4_denselayer2_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer2_conv1   L__self___features_denseblock4_denselayer2_conv1            (l__self___features_denseblock4_denselayer2_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer2_norm2   L__self___features_denseblock4_denselayer2_norm2            (l__self___features_denseblock4_denselayer2_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer2_relu2   L__self___features_denseblock4_denselayer2_relu2            (l__self___features_denseblock4_denselayer2_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer2_conv2   L__self___features_denseblock4_denselayer2_conv2            (l__self___features_denseblock4_denselayer2_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_47                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             {}
call_module    l__self___features_denseblock4_denselayer3_norm1   L__self___features_denseblock4_denselayer3_norm1            (cat_47,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer3_relu1   L__self___features_denseblock4_denselayer3_relu1            (l__self___features_denseblock4_denselayer3_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer3_conv1   L__self___features_denseblock4_denselayer3_conv1            (l__self___features_denseblock4_denselayer3_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer3_norm2   L__self___features_denseblock4_denselayer3_norm2            (l__self___features_denseblock4_denselayer3_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer3_relu2   L__self___features_denseblock4_denselayer3_relu2            (l__self___features_denseblock4_denselayer3_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer3_conv2   L__self___features_denseblock4_denselayer3_conv2            (l__self___features_denseblock4_denselayer3_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_48                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           {}
call_module    l__self___features_denseblock4_denselayer4_norm1   L__self___features_denseblock4_denselayer4_norm1            (cat_48,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer4_relu1   L__self___features_denseblock4_denselayer4_relu1            (l__self___features_denseblock4_denselayer4_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer4_conv1   L__self___features_denseblock4_denselayer4_conv1            (l__self___features_denseblock4_denselayer4_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer4_norm2   L__self___features_denseblock4_denselayer4_norm2            (l__self___features_denseblock4_denselayer4_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer4_relu2   L__self___features_denseblock4_denselayer4_relu2            (l__self___features_denseblock4_denselayer4_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer4_conv2   L__self___features_denseblock4_denselayer4_conv2            (l__self___features_denseblock4_denselayer4_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_49                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         {}
call_module    l__self___features_denseblock4_denselayer5_norm1   L__self___features_denseblock4_denselayer5_norm1            (cat_49,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer5_relu1   L__self___features_denseblock4_denselayer5_relu1            (l__self___features_denseblock4_denselayer5_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer5_conv1   L__self___features_denseblock4_denselayer5_conv1            (l__self___features_denseblock4_denselayer5_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer5_norm2   L__self___features_denseblock4_denselayer5_norm2            (l__self___features_denseblock4_denselayer5_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer5_relu2   L__self___features_denseblock4_denselayer5_relu2            (l__self___features_denseblock4_denselayer5_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer5_conv2   L__self___features_denseblock4_denselayer5_conv2            (l__self___features_denseblock4_denselayer5_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_50                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer6_norm1   L__self___features_denseblock4_denselayer6_norm1            (cat_50,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer6_relu1   L__self___features_denseblock4_denselayer6_relu1            (l__self___features_denseblock4_denselayer6_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer6_conv1   L__self___features_denseblock4_denselayer6_conv1            (l__self___features_denseblock4_denselayer6_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer6_norm2   L__self___features_denseblock4_denselayer6_norm2            (l__self___features_denseblock4_denselayer6_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer6_relu2   L__self___features_denseblock4_denselayer6_relu2            (l__self___features_denseblock4_denselayer6_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer6_conv2   L__self___features_denseblock4_denselayer6_conv2            (l__self___features_denseblock4_denselayer6_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_51                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     {}
call_module    l__self___features_denseblock4_denselayer7_norm1   L__self___features_denseblock4_denselayer7_norm1            (cat_51,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer7_relu1   L__self___features_denseblock4_denselayer7_relu1            (l__self___features_denseblock4_denselayer7_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer7_conv1   L__self___features_denseblock4_denselayer7_conv1            (l__self___features_denseblock4_denselayer7_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer7_norm2   L__self___features_denseblock4_denselayer7_norm2            (l__self___features_denseblock4_denselayer7_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer7_relu2   L__self___features_denseblock4_denselayer7_relu2            (l__self___features_denseblock4_denselayer7_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer7_conv2   L__self___features_denseblock4_denselayer7_conv2            (l__self___features_denseblock4_denselayer7_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_52                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock4_denselayer8_norm1   L__self___features_denseblock4_denselayer8_norm1            (cat_52,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer8_relu1   L__self___features_denseblock4_denselayer8_relu1            (l__self___features_denseblock4_denselayer8_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer8_conv1   L__self___features_denseblock4_denselayer8_conv1            (l__self___features_denseblock4_denselayer8_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer8_norm2   L__self___features_denseblock4_denselayer8_norm2            (l__self___features_denseblock4_denselayer8_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer8_relu2   L__self___features_denseblock4_denselayer8_relu2            (l__self___features_denseblock4_denselayer8_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer8_conv2   L__self___features_denseblock4_denselayer8_conv2            (l__self___features_denseblock4_denselayer8_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_53                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 {}
call_module    l__self___features_denseblock4_denselayer9_norm1   L__self___features_denseblock4_denselayer9_norm1            (cat_53,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer9_relu1   L__self___features_denseblock4_denselayer9_relu1            (l__self___features_denseblock4_denselayer9_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer9_conv1   L__self___features_denseblock4_denselayer9_conv1            (l__self___features_denseblock4_denselayer9_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer9_norm2   L__self___features_denseblock4_denselayer9_norm2            (l__self___features_denseblock4_denselayer9_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer9_relu2   L__self___features_denseblock4_denselayer9_relu2            (l__self___features_denseblock4_denselayer9_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_module    l__self___features_denseblock4_denselayer9_conv2   L__self___features_denseblock4_denselayer9_conv2            (l__self___features_denseblock4_denselayer9_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        {}
call_function  cat_54                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2, l__self___features_denseblock4_denselayer9_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               {}
call_module    l__self___features_denseblock4_denselayer10_norm1  L__self___features_denseblock4_denselayer10_norm1           (cat_54,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer10_relu1  L__self___features_denseblock4_denselayer10_relu1           (l__self___features_denseblock4_denselayer10_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer10_conv1  L__self___features_denseblock4_denselayer10_conv1           (l__self___features_denseblock4_denselayer10_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer10_norm2  L__self___features_denseblock4_denselayer10_norm2           (l__self___features_denseblock4_denselayer10_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer10_relu2  L__self___features_denseblock4_denselayer10_relu2           (l__self___features_denseblock4_denselayer10_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer10_conv2  L__self___features_denseblock4_denselayer10_conv2           (l__self___features_denseblock4_denselayer10_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_55                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2, l__self___features_denseblock4_denselayer9_conv2, l__self___features_denseblock4_denselayer10_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            {}
call_module    l__self___features_denseblock4_denselayer11_norm1  L__self___features_denseblock4_denselayer11_norm1           (cat_55,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer11_relu1  L__self___features_denseblock4_denselayer11_relu1           (l__self___features_denseblock4_denselayer11_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer11_conv1  L__self___features_denseblock4_denselayer11_conv1           (l__self___features_denseblock4_denselayer11_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer11_norm2  L__self___features_denseblock4_denselayer11_norm2           (l__self___features_denseblock4_denselayer11_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer11_relu2  L__self___features_denseblock4_denselayer11_relu2           (l__self___features_denseblock4_denselayer11_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer11_conv2  L__self___features_denseblock4_denselayer11_conv2           (l__self___features_denseblock4_denselayer11_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_56                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2, l__self___features_denseblock4_denselayer9_conv2, l__self___features_denseblock4_denselayer10_conv2, l__self___features_denseblock4_denselayer11_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         {}
call_module    l__self___features_denseblock4_denselayer12_norm1  L__self___features_denseblock4_denselayer12_norm1           (cat_56,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer12_relu1  L__self___features_denseblock4_denselayer12_relu1           (l__self___features_denseblock4_denselayer12_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer12_conv1  L__self___features_denseblock4_denselayer12_conv1           (l__self___features_denseblock4_denselayer12_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer12_norm2  L__self___features_denseblock4_denselayer12_norm2           (l__self___features_denseblock4_denselayer12_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer12_relu2  L__self___features_denseblock4_denselayer12_relu2           (l__self___features_denseblock4_denselayer12_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer12_conv2  L__self___features_denseblock4_denselayer12_conv2           (l__self___features_denseblock4_denselayer12_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_57                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2, l__self___features_denseblock4_denselayer9_conv2, l__self___features_denseblock4_denselayer10_conv2, l__self___features_denseblock4_denselayer11_conv2, l__self___features_denseblock4_denselayer12_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      {}
call_module    l__self___features_denseblock4_denselayer13_norm1  L__self___features_denseblock4_denselayer13_norm1           (cat_57,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer13_relu1  L__self___features_denseblock4_denselayer13_relu1           (l__self___features_denseblock4_denselayer13_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer13_conv1  L__self___features_denseblock4_denselayer13_conv1           (l__self___features_denseblock4_denselayer13_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer13_norm2  L__self___features_denseblock4_denselayer13_norm2           (l__self___features_denseblock4_denselayer13_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer13_relu2  L__self___features_denseblock4_denselayer13_relu2           (l__self___features_denseblock4_denselayer13_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer13_conv2  L__self___features_denseblock4_denselayer13_conv2           (l__self___features_denseblock4_denselayer13_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_58                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2, l__self___features_denseblock4_denselayer9_conv2, l__self___features_denseblock4_denselayer10_conv2, l__self___features_denseblock4_denselayer11_conv2, l__self___features_denseblock4_denselayer12_conv2, l__self___features_denseblock4_denselayer13_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___features_denseblock4_denselayer14_norm1  L__self___features_denseblock4_denselayer14_norm1           (cat_58,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer14_relu1  L__self___features_denseblock4_denselayer14_relu1           (l__self___features_denseblock4_denselayer14_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer14_conv1  L__self___features_denseblock4_denselayer14_conv1           (l__self___features_denseblock4_denselayer14_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer14_norm2  L__self___features_denseblock4_denselayer14_norm2           (l__self___features_denseblock4_denselayer14_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer14_relu2  L__self___features_denseblock4_denselayer14_relu2           (l__self___features_denseblock4_denselayer14_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer14_conv2  L__self___features_denseblock4_denselayer14_conv2           (l__self___features_denseblock4_denselayer14_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_59                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2, l__self___features_denseblock4_denselayer9_conv2, l__self___features_denseblock4_denselayer10_conv2, l__self___features_denseblock4_denselayer11_conv2, l__self___features_denseblock4_denselayer12_conv2, l__self___features_denseblock4_denselayer13_conv2, l__self___features_denseblock4_denselayer14_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                {}
call_module    l__self___features_denseblock4_denselayer15_norm1  L__self___features_denseblock4_denselayer15_norm1           (cat_59,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer15_relu1  L__self___features_denseblock4_denselayer15_relu1           (l__self___features_denseblock4_denselayer15_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer15_conv1  L__self___features_denseblock4_denselayer15_conv1           (l__self___features_denseblock4_denselayer15_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer15_norm2  L__self___features_denseblock4_denselayer15_norm2           (l__self___features_denseblock4_denselayer15_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer15_relu2  L__self___features_denseblock4_denselayer15_relu2           (l__self___features_denseblock4_denselayer15_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer15_conv2  L__self___features_denseblock4_denselayer15_conv2           (l__self___features_denseblock4_denselayer15_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_60                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2, l__self___features_denseblock4_denselayer9_conv2, l__self___features_denseblock4_denselayer10_conv2, l__self___features_denseblock4_denselayer11_conv2, l__self___features_denseblock4_denselayer12_conv2, l__self___features_denseblock4_denselayer13_conv2, l__self___features_denseblock4_denselayer14_conv2, l__self___features_denseblock4_denselayer15_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                             {}
call_module    l__self___features_denseblock4_denselayer16_norm1  L__self___features_denseblock4_denselayer16_norm1           (cat_60,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_module    l__self___features_denseblock4_denselayer16_relu1  L__self___features_denseblock4_denselayer16_relu1           (l__self___features_denseblock4_denselayer16_norm1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer16_conv1  L__self___features_denseblock4_denselayer16_conv1           (l__self___features_denseblock4_denselayer16_relu1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer16_norm2  L__self___features_denseblock4_denselayer16_norm2           (l__self___features_denseblock4_denselayer16_conv1,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer16_relu2  L__self___features_denseblock4_denselayer16_relu2           (l__self___features_denseblock4_denselayer16_norm2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_module    l__self___features_denseblock4_denselayer16_conv2  L__self___features_denseblock4_denselayer16_conv2           (l__self___features_denseblock4_denselayer16_relu2,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       {}
call_function  cat_61                                             <built-in method cat of type object at 0x7f3a8f700a40>      ([l__self___features_transition3_pool, l__self___features_denseblock4_denselayer1_conv2, l__self___features_denseblock4_denselayer2_conv2, l__self___features_denseblock4_denselayer3_conv2, l__self___features_denseblock4_denselayer4_conv2, l__self___features_denseblock4_denselayer5_conv2, l__self___features_denseblock4_denselayer6_conv2, l__self___features_denseblock4_denselayer7_conv2, l__self___features_denseblock4_denselayer8_conv2, l__self___features_denseblock4_denselayer9_conv2, l__self___features_denseblock4_denselayer10_conv2, l__self___features_denseblock4_denselayer11_conv2, l__self___features_denseblock4_denselayer12_conv2, l__self___features_denseblock4_denselayer13_conv2, l__self___features_denseblock4_denselayer14_conv2, l__self___features_denseblock4_denselayer15_conv2, l__self___features_denseblock4_denselayer16_conv2], 1)                                                                                                                                                                                                                                                                                                                                                                                                                          {}
call_module    l__self___features_norm5                           L__self___features_norm5                                    (cat_61,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  {}
call_function  relu                                               <function relu at 0x7f39f90268c0>                           (l__self___features_norm5,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                {'inplace': True}
call_function  adaptive_avg_pool2d                                <function adaptive_avg_pool2d at 0x7f39f90263b0>            (relu, (1, 1))                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             {}
call_function  flatten                                            <built-in method flatten of type object at 0x7f3a8f700a40>  (adaptive_avg_pool2d, 1)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   {}
call_module    l__self___classifier                               L__self___classifier                                        (flatten,)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 {}
output         output                                             output                                                      ((l__self___classifier,),)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 {}

tensor([[ 0.0614, -0.4023, -0.2792,  ..., -0.5549,  0.0976, -0.0634],
        [-0.2032, -0.2706, -0.0935,  ..., -0.4815,  0.0758, -0.1038],
        [ 0.0637, -0.3492, -0.1492,  ..., -0.4841,  0.1776, -0.0723],
        ...,
        [-0.1050, -0.3393,  0.0092,  ..., -0.4862,  0.0555, -0.1058],
        [ 0.0018, -0.2431, -0.1656,  ..., -0.5072,  0.0977, -0.1387],
        [ 0.1192, -0.3563, -0.1147,  ..., -0.4839,  0.1770, -0.0659]],
       device='cuda:0', grad_fn=<AddmmBackward0>)

使用我们的自定义后端,我们现在可以看到 TorchDynamo 如何处理 依赖于数据的控制流。考虑下面的函数,其中行 `if

b.sum()

<

0` 是数据相关控制流的源。

def bar(a, b):
    x = a / (torch.abs(a) + 1)
    if b.sum() < 0:
        b = b * -1
    return x * b

opt_bar = torch.compile(bar, backend=custom_backend)
inp1 = torch.randn(10)
inp2 = torch.randn(10)
opt_bar(inp1, inp2)
opt_bar(inp1, -inp2)
custom backend called with FX graph:
opcode         name     target                                                  args              kwargs
-------------  -------  ------------------------------------------------------  ----------------  --------
placeholder    l_a_     L_a_                                                    ()                {}
placeholder    l_b_     L_b_                                                    ()                {}
call_function  abs_1    <built-in method abs of type object at 0x7f3a8f700a40>  (l_a_,)           {}
call_function  add      <built-in function add>                                 (abs_1, 1)        {}
call_function  truediv  <built-in function truediv>                             (l_a_, add)       {}
call_method    sum_1    sum                                                     (l_b_,)           {}
call_function  lt       <built-in function lt>                                  (sum_1, 0)        {}
output         output   output                                                  ((truediv, lt),)  {}
custom backend called with FX graph:
opcode         name    target                   args          kwargs
-------------  ------  -----------------------  ------------  --------
placeholder    l_b_    L_b_                     ()            {}
placeholder    l_x_    L_x_                     ()            {}
call_function  mul     <built-in function mul>  (l_x_, l_b_)  {}
output         output  output                   ((mul,),)     {}
custom backend called with FX graph:
opcode         name    target                   args         kwargs
-------------  ------  -----------------------  -----------  --------
placeholder    l_b_    L_b_                     ()           {}
placeholder    l_x_    L_x_                     ()           {}
call_function  mul     <built-in function mul>  (l_b_, -1)   {}
call_function  mul_1   <built-in function mul>  (l_x_, mul)  {}
output         output  output                   ((mul_1,),)  {}

tensor([-0.0176,  1.0753,  0.0282,  0.0756, -0.0176,  0.0633, -0.9161,  0.1333,
        -0.1971, -0.3406])

输出显示 TorchDynamo 提取了 3 个不同的 FX 图表 对应于以下代码(顺序可能与上面的输出不同):

  1. `x

=

a

/

(torch.abs(a)

+

1)2.b

=

b

*

-1;

返回

x

*

b3.返回

x

*

b`

当 TorchDynamo 遇到不支持的 Python 功能(例如依赖于数据的控制流)时,它会中断计算图,让默认的 Python 解释器处理不支持的代码,然后继续捕获图。

让’s 通过示例研究 TorchDynamo 如何单步执行 bar 。 If `b.sum()

<

0,那么 TorchDynamo 将运行图 1,让 Python 确定条件的结果,然后运行 图 2。另一方面, ifnot

b.sum() \ n <

0` ,然后 TorchDynamo 将运行图 1,让 Python 确定条件的结果,然后 运行图 3。

这凸显了 TorchDynamo 和之前的 PyTorch 编译器解决方案之间的主要区别。当遇到不受支持的 Python 功能时, 以前的解决方案要么引发错误,要么默默失败。 另一方面,TorchDynamo 会破坏计算图。

我们可以通过使用 torch._dynamo.explain 来查看 TorchDynamo 在哪里破坏图表 :

# Reset since we are using a different backend.
torch._dynamo.reset()
explain_output = torch._dynamo.explain(bar)(torch.randn(10), torch.randn(10))
print(explain_output)
Graph Count: 2
Graph Break Count: 1
Op Count: 6
Break Reasons:
  Break Reason 1:
    Reason: generic_jump TensorVariable()
    User Stack:
      <FrameSummary file /var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py, line 434 in bar>
Ops per Graph:
  Ops 1:
    <built-in method abs of type object at 0x7f3a8f700a40>
    <built-in function add>
    <built-in function truediv>
    <built-in function lt>
  Ops 2:
    <built-in function mul>
    <built-in function mul>
Out Guards:
  Guard 1:
    Name: ''
    Source: global
    Create Function: GRAD_MODE
    Guard Types: ['GRAD_MODE']
    Code List: ['___is_grad_enabled()']
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 2:
    Name: "L['b']"
    Source: local
    Create Function: TENSOR_MATCH
    Guard Types: ['TENSOR_MATCH']
    Code List: ["hasattr(L['b'], '_dynamo_dynamic_indices') == False"]
    Object Weakref: <weakref at 0x7f3a0a0040e0; dead>
    Guarded Class Weakref: <weakref at 0x7f39f9543510; to 'torch._C._TensorMeta' at 0x609c370 (Tensor)>
  Guard 3:
    Name: ''
    Source: shape_env
    Create Function: SHAPE_ENV
    Guard Types: None
    Code List: None
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 4:
    Name: ''
    Source: global
    Create Function: DETERMINISTIC_ALGORITHMS
    Guard Types: ['DETERMINISTIC_ALGORITHMS']
    Code List: ['not ___are_deterministic_algorithms_enabled()']
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 5:
    Name: "L['a']"
    Source: local
    Create Function: TENSOR_MATCH
    Guard Types: ['TENSOR_MATCH']
    Code List: ["hasattr(L['a'], '_dynamo_dynamic_indices') == False"]
    Object Weakref: <weakref at 0x7f3918677100; dead>
    Guarded Class Weakref: <weakref at 0x7f39f9543510; to 'torch._C._TensorMeta' at 0x609c370 (Tensor)>
  Guard 6:
    Name: ''
    Source: global
    Create Function: TORCH_FUNCTION_STATE
    Guard Types: ['TORCH_FUNCTION_STATE']
    Code List: ['___is_torch_function_enabled()']
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 7:
    Name: ''
    Source: global
    Create Function: DEFAULT_DEVICE
    Guard Types: ['DEFAULT_DEVICE']
    Code List: ['utils_device.CURRENT_DEVICE == None']
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 8:
    Name: "G['torch']"
    Source: global
    Create Function: FUNCTION_MATCH
    Guard Types: None
    Code List: None
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 9:
    Name: ''
    Source: global
    Create Function: GRAD_MODE
    Guard Types: ['GRAD_MODE']
    Code List: ['___is_grad_enabled()']
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 10:
    Name: "L['b']"
    Source: local
    Create Function: TENSOR_MATCH
    Guard Types: ['TENSOR_MATCH']
    Code List: ["hasattr(L['b'], '_dynamo_dynamic_indices') == False"]
    Object Weakref: <weakref at 0x7f3a0a0040e0; dead>
    Guarded Class Weakref: <weakref at 0x7f39f9543510; to 'torch._C._TensorMeta' at 0x609c370 (Tensor)>
  Guard 11:
    Name: ''
    Source: shape_env
    Create Function: SHAPE_ENV
    Guard Types: None
    Code List: None
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 12:
    Name: ''
    Source: global
    Create Function: DETERMINISTIC_ALGORITHMS
    Guard Types: ['DETERMINISTIC_ALGORITHMS']
    Code List: ['not ___are_deterministic_algorithms_enabled()']
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 13:
    Name: ''
    Source: global
    Create Function: TORCH_FUNCTION_STATE
    Guard Types: ['TORCH_FUNCTION_STATE']
    Code List: ['___is_torch_function_enabled()']
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 14:
    Name: ''
    Source: global
    Create Function: DEFAULT_DEVICE
    Guard Types: ['DEFAULT_DEVICE']
    Code List: ['utils_device.CURRENT_DEVICE == None']
    Object Weakref: None
    Guarded Class Weakref: None
  Guard 15:
    Name: "L['x']"
    Source: local
    Create Function: TENSOR_MATCH
    Guard Types: ['TENSOR_MATCH']
    Code List: ["hasattr(L['x'], '_dynamo_dynamic_indices') == False"]
    Object Weakref: <weakref at 0x7f3970129710; dead>
    Guarded Class Weakref: <weakref at 0x7f39f9543510; to 'torch._C._TensorMeta' at 0x609c370 (Tensor)>
Compile Times: TorchDynamo compilation metrics:
Function                         Runtimes (s)
-------------------------------  --------------
_compile.<locals>.compile_inner  0.0139, 0.0072
OutputGraph.call_user_compiler   0.0010, 0.0000

为了最大限度地提高加速速度,应限制图形中断。 我们可以强制 TorchDynamo 在使用 遇到的第一个图形 中断时引发错误 fullgraph=True :

opt_bar = torch.compile(bar, fullgraph=True)
try:
    opt_bar(torch.randn(10), torch.randn(10))
except:
    tb.print_exc()
Traceback (most recent call last):
  File "/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py", line 482, in <module>
    opt_bar(torch.randn(10), torch.randn(10))
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 328, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 490, in catch_errors
    return callback(frame, cache_entry, hooks, frame_state)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 133, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 389, in _convert_frame_assert
    return _compile(
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 569, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 189, in time_wrapper
    r = func(*args, **kwargs)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 491, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/bytecode_transformation.py", line 1028, in transform_code_object
    transformations(instructions, code_options)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/convert_frame.py", line 458, in transform
    tracer.run()
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 2074, in run
    super().run()
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 724, in run
    and self.step()
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 688, in step
    getattr(self, inst.opname)(inst)
  File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_dynamo/symbolic_convert.py", line 370, in inner
    raise exc.UserError(
torch._dynamo.exc.UserError: Dynamic control flow is not supported at the moment. Please use functorch.experimental.control_flow.cond to explicitly capture the control flow

from user code:
   File "/var/lib/jenkins/workspace/intermediate_source/torch_compile_tutorial.py", line 434, in bar
    if b.sum() < 0:

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True

下面,我们将演示 TorchDynamo 不会破坏我们上面用于演示加速的模型上的图表。

opt_model = torch.compile(init_model(), fullgraph=True)
print(opt_model(generate_data(16)[0]))
tensor([[ 0.1346,  0.1981, -0.2058,  ...,  0.1868, -0.1626, -0.1644],
        [ 0.2679,  0.2347, -0.1904,  ...,  0.2167, -0.0060,  0.0307],
        [ 0.0166,  0.2182, -0.1113,  ...,  0.1708, -0.1683, -0.0637],
        ...,
        [ 0.0808,  0.2680, -0.1887,  ...,  0.0378, -0.2078, -0.1444],
        [-0.0211,  0.0857, -0.2459,  ...,  0.1863, -0.1282, -0.0283],
        [-0.0388,  0.0728, -0.1961,  ...,  0.0860, -0.2200, -0.1485]],
       device='cuda:0', grad_fn=<CompiledFunctionBackward>)

我们可以使用 torch.export (来自 PyTorch 2.1+)从输入 PyTorch 程序中提取单个可导出 FX 图表。导出的图表旨在 在不同的(即无 Python)环境中运行。一个重要的限制是 torch.export 不支持图表中断。请查看 本教程 了解有关 torch.export的更多详细信息 。

结论

在本教程中,我们介绍了 torch.compile ,介绍了 基本用法、演示了比 Eager 模式更快的速度、与之前 PyTorch 编译器解决方案进行比较,并简要研究了 TorchDynamo 及其与 FX 图的交互 。我们希望您 尝试一下torch.compile

脚本的总运行时间: (6 分 50.110 秒)



回到顶部