Recently, several economies with interest rates close to zero have received large capital inflows while their central banks accumulated large foreign reserves. Concurrently, significant deviations from covered interest parity have appeared. We show that, with limited international arbitrage, a central bank's pursuit of an exchange rate policy at the ZLB can explain these facts. We provide a measure of the costs associated with this policy and show they can be sizable. Changes in external conditions that increase capital inflows are detrimental, even when they are beneficial away from the ZLB. Negative nominal rates and capital controls can reduce the costs.