Three-dimensional topological insulators (3D TIs) with extraordinary electronic states have attracted intense research interest these years. For an ideal 3D TI, bulk states feature an energy gap like an ordinary insulator, while surface states are characterized by a linear Dirac-like dispersion energy band with spin texture locked helically to momentum, resulting in an insulating bulk and metallic surfaces. The most studied 3D TIs include Bi₂Se₃, Bi₂Te₃ and Sb₂Te₃ binary compounds whose unit cell consists of three quintuple layers (QLs) bonded by Van der Waals force. This dissertation presents research works carried out on two 3D TI thin film structures fabricated by the molecular beam epitaxy technique. One is a Bi₂Te₃ thin film with Fe heavy doping; the other is superconductin...[ Read more ]

Three-dimensional topological insulators (3D TIs) with extraordinary electronic states have attracted intense research interest these years. For an ideal 3D TI, bulk states feature an energy gap like an ordinary insulator, while surface states are characterized by a linear Dirac-like dispersion energy band with spin texture locked helically to momentum, resulting in an insulating bulk and metallic surfaces. The most studied 3D TIs include Bi₂Se₃, Bi₂Te₃ and Sb₂Te₃ binary compounds whose unit cell consists of three quintuple layers (QLs) bonded by Van der Waals force. This dissertation presents research works carried out on two 3D TI thin film structures fabricated by the molecular beam epitaxy technique. One is a Bi₂Te₃ thin film with Fe heavy doping; the other is superconducting TI / iron-based parent compound heterostructures (Bi₂Te₃/FeTe and Sb₂Te₃/FeTe).

The fabrication of the Bi₂Te₃ thin film with Fe heavy doping was aimed to study if a certain Fe doping concentration in Bi₂Te₃ could make it superconducting, which is based on the thought that the observed interfacial SC at the Bi₂Te₃/FeTe heterostructure may be caused by forming a superconducting Bi₂Te₃:Fe layer at the interface of the heterostructure at a certain doping level due to Fe diffusion. In the Fe doped Bi₂Te₃ thin film, the dopant concentration is monotonically increased up to about 6.9% along the growth direction. Two types of unexpected Fe-Te nanostructures - one is FeTe thin layer formed near the surface and the other is FeTe₂ nano-rod embedded in the Bi₂Te₃ layer - were found by scanning electron microscope, X-ray diffraction and transmission electron microscope. The resistance versus temperature curve of this sample displays a superconducting transition at about 12.3 K. The magnetic-field dependences of the onset temperature of the detected drop of resistance and the critical fields extrapolated from Ginzburg-Landau equations show a trend similar to that of a superconducting Bi₂Te₃ (7 QL)/FeTe heterostructure, indicating they share the same origin of the observed superconductivity (SC). The formation mechanisms of the two types Fe-Te nanostructures are addressed. This study provides an unusual approach to synthesizing nanostructures or heterostructures via heavy doping if the dopant element is strongly reactive with an element in the host matrix. More importantly, since the observed SC in this Fe doped Bi₂Te₃ thin film cannot reach zero resistance as the FeTe nanostructure is not continuous across the thin film, together with the fact that its neighboring Bi₂Te₃ layer is likely lightly doped with Fe, however, the latter is not superconducting, we thus can rule out the possibility that the observed SC in this system is simply due to Fe doping in the Bi₂Te₃ layer.

Fabrication of Sb₂Te₃/FeTe heterostructures is aimed at understanding the superconducting mechanism in TI/FeTe systems. First, this study confirmed that the Sb₂Te₃/FeTe heterostruture indeed shows a superconducting behavior and the highest transition temperature is around 12.3 K. The superconducting properties of a Sb₂Te₃/FeTe heterostructure have been further analyzed by Ginzburg-Landau theory and Berezinskii-Kosterlitz-Thouless theoretical model, which confirm that the observed SC has a two-dimensional (2D) nature. The crystal structure analysis was carried out by a spherical-aberration-corrected scanning transmission electron microscope, showing atomically sharp interfaces between the Sb₂Te₃ domains and the FeTe layer. Several possible hypotheses have been proposed and tested for explaining the observed SC. The results show that the reduction of excess Fe in the FeTe layer increases its fluctuation of the antiferromagnetic (AFM) order and makes the heterostructure easier to become superconducting. Also, we found that increasing the TI thickness could improve the quality of the interfacial SC of this heterostructure system. In addition, the interfacial SC of Sb₂Te₃/FeTe heterostructure was found to have a wider-ranging TI-layer thickness dependence than that of the Bi₂Te₃/FeTe heterostructure, which is believed to be attributed to the much higher bulk conductivity of Sb₂Te₂ that enhances stronger indirect coupling between its top and bottom topological surface states (TSSs). On the other hand, the electrical transport results of a Bi/FeTe and a Sb/FeTe heterostructure indicate that spin-orbit coupling (SOC) alone is not able to induce the interfacial SC observed in TI/Fe_1+yTe heterostructures. We also demonstrate that the deposition of a TI layer on top of a FeTe layer does not significantly modifies the AFM order of the FeTe layer, which is revealed by comparing the AFM phase transition temperatures of a pure FeTe (8 nm) thin film and a superconducting Sb₂Te₃ (24 QLs) / FeTe (8 nm) heterostructure. The above results provide the evidence of the interplay among the fluctuation of AFM order, itinerant carries from the TSSs and the induced interfacial SC of the TI/Fe_1+yTe heterostructure system. Finally, based on the experimental results obtained in this thesis research, a possible pairing mechanism has been proposed to correlate with the underlying mechanism of the 2D SC of the TI/Fe_1+yTe heterostructure system, in which the unconventional pairing is believed to be originated from the even stronger spin fluctuation caused by the hybridization of the local spin moments in Fe_1+yTe and the 2D itinerant electrons from the TSSs of the TI layer.