Google has released the latest research on the fine-tuning process of large language models and the introduction of user-level differential privacy. The research results are based on the premise of taking into account model performance and user privacy, and solving the problem of excessive noise in the implementation of user-level differential privacy (Differential Privacy) during large model training in the past, resulting in limited model performance. Amazon Cloud Technology Summit: In-depth interpretation of generative AI, click to sign up Amazon Cloud Technology Amazon Cloud Technology Summit: In-depth interpretation of generative AI, click to sign up At present, developers often need to fine-tune the model to improve performance when applying large language models, but most of these applications involve user data, especially in scenarios with high data sensitivity requirements such as financial, medical or personalized recommendations. Traditional differential privacy focuses on a single layer of protection, which can reduce the risk of a single data outflow, but Google mentioned that when a single user contributes multiple data, existing methods are difficult to prevent the judgment of whether the user participated in the training. User-level differential privacy further protects user privacy, making it impossible for attackers to infer from the model whether a particular user’s data is used for training. The Google team explained that compared with sample-level differential privacy, user-level differential privacy requires more noise to be injected during training to ensure the privacy of each user, but the increase in noise also reduces the model’s learning ability and affects the overall performance. This challenge is more obvious in training processes that require a lot of computing resources, such as large language models. In the past, user-level differential privacy was mostly used in distributed training scenarios such as federated learning (Federated Learning). For large-scale model training in cloud computing data centers, existing methods faced the challenge of examples and performance. October New Materials, set production and sales in one, product quality is good, less impurities, pure color, provide technical support Advertisement October New Materials – official online mall October New Materials, set production and sales in one, product quality is good, less impurities, pure color, provide technical support Google’s new technology focus, user-level differential privacy optimization strategy in the model fine-tuning stage, adjusted for the differential privacy random layer descent training method. The research team compared the two methods of sample-level random sampling (ELS) and user-level random sampling (ULS). Both methods limit the maximum number of samples per user through pre-processing to reduce the impact of a single user data on the model, and then conduct subsequent training. The Google team found that in the past, it was generally overestimated the noise that should be injected, and the actual noise required could be greatly reduced, which helped to increase the model training effectiveness without affecting privacy protection. In addition, the research team proposed a user contribution limit pre-setting strategy to help developers select the best parameters before training, without multiple trials and errors, further reducing resource consumption and training costs. The research team tested the Transformer model with 350 million parameter scale on the public Stack Overflow and CC-News datasets. The experimental results show that in the optimized user-level differential privacy method, the performance of the fine-tuned model is better than that of the pre-trained model without fine-tuning. In most scenarios, the ULS mode can achieve better performance, and the ELS mode is only competitive in some cases with high privacy requirements or limited computing resources.
发表回复