View Item 
        •   Utrecht University Student Theses Repository Home
        • UU Theses Repository
        • Theses
        • View Item
        •   Utrecht University Student Theses Repository Home
        • UU Theses Repository
        • Theses
        • View Item
        JavaScript is disabled for your browser. Some features of this site may not work without it.

        Browse

        All of UU Student Theses RepositoryBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

        How can the knowledge of lottery tickets be used to adapt SynFlow to early pruning after pre-training?

        Thumbnail
        View/Open
        Bolwerk6231489.pdf (590.1Kb)
        Publication date
        2021
        Author
        Bolwerk, C.J.
        Metadata
        Show full item record
        Summary
        This research focuses on adapting an early-pruning algorithm called SynFlow to prune after pre-training. Adapting in this way was inspired by work around the Lottery Ticket Hypothesis by Frankle et al. We show that the instability analysis is a way to determine the stability of a network at an iteration k, after which post-training a subnetwork will result in it matching a full network’s accuracy. While this still reduces complexity and training time during training as the original SynFlow, it also improves accuracy significantly. In the current training setup, a full ResNet-20 network is able to achieve accuracies as high as 90.14%. SynFlow on average gets up to a 80% accuracy, sometimes reaching the low 80 percentages. This research presents AdSynFlow, where we consistently achieve higher accuracies than SynFlow, with a high of 85.27%. Though not yet matching the full network, AdSynFlow promises a bright future for early pruning methods.
        URI
        https://studenttheses.uu.nl/handle/20.500.12932/40695
        Collections
        • Theses
        Utrecht university logo