In view of the significant influences of multi-source uncertainties on structural safety, which generally exist in practical engineering (such as the dispersion of material, the uncertainty of external load and the error of processing technology), more academic research and engineering applications had paid attention to uncertainty in recent years. However, due to the complexity of the structural problems, there may be multiple uncertain parameters. Traditional methods of optimal design by single-source uncertainty model, particularly the one derived from probability theory, may no longer be feasible. This paper investigates a new formulation and numerical solution of reliability-based design optimization (RBDO) of structures exhibiting random and uncertain-but-bounded (interval and convex) mixed uncertainties. Combined with the non-probabilistic set-theory convex model and the classical probabilistic approach, the mathematical definition of hybrid reliability is firstly presented for a quantified measure of the safety margin. The reliability-based optimization incorporating such mixed reliability constraints is then formulated. The PSO algorithm is employed to improve the convergence and the stability in seeking the optimal global solution. Additionally, by introducing the general concept of the safety factor, the compatibility between the proposed hybrid RBDO technique and the safety factor-based model is further discussed. By virtue of the above two methods, two numerical examples of typical components (the cantilever structure and the truss structure) as well as one complex engineering example (the hypersonic wing structure) are performed, subjected to the strength or stiffness criteria. The accuracy and effectiveness of the present method are then demonstrated.