This repository was archived by the owner on Aug 6, 2025. It is now read-only.

This repository was archived by the owner on Aug 6, 2025. It is now read-only.

Activations in AdaLN #113

Open

opened

I wonder if there's a specific reason for having an activation function before the linear projection in AdaLN. Would it be better to have particular activations after the linear projection in addition to that? E.g., sigmoid for the gate, positive for scale, identity for shift.

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests