The store would function much like Amazon itself, according to a report in the Wall Street Journal.
Customers could pick up online orders, have items delivered the same day within New York and exchange or return products.
Amazon could also sell its own-brand products, such as the Kindle e-reader and tablet range, smartphones and its set-top box.
The store will be based close to the Empire State Building and popular department store Macy’s, according to the report.
This is not the first time Amazon has tried to move into the offline world. The company opened a pop-up shop in San Francisco earlier this year to teach customers about Amazon Web Services.
And in November last year, the company opened a small pop-up shop inside a San Francisco mall, selling its own devices and accessories.
Amazon has yet to comment on the news, but it is thought the company will open other stores in US cities if the experiment goes well.