The state of WI has banned any shipments of black walnuts coming into the state due to thousand cankers disease. I know as the nursery I am at had to start our own black walnuts from seed as nobody will ship us bare-root black walnut liners from out of state.
Hickory nuts may grow for you. In the wild they are found just North of LaCrosse, WI. That seems as far north as they got as a native plant. However I have shipped BB hickory nut trees to nurseries in Minneapolis and they grow just fine there. Also shipped one to Sioux Falls and found it survived! Watch your seed source as you want one that has genetics from a northern source if possible.
American hazelnut is fine in Zone 4 and should be no problem to grow.
Pecans? We tried growing them in SE WI at the nursery I work at using seed from Clinton, IA (reported to be the most northern range of wild pecan found). The trees grew great! Due to lack of sales and finding that they never fruited we stopped growing them. Foolish decision. Old trees left in the field did bear but not at an early age. Eventually they bore their delicious tan nuts that are so sweet but smaller in length than most commercial varieties. Not sure where you could find a source for those hardy pecans anymore. Nor if they would survive zone 4. We are in northern zone 5 where I am in SE Wisconsin.
Butternut grows here but rarely found. I recall some disease issue struck when I was a young boy and wiped most of them out. I see a large butternut exists in the town of Ripon WI but out in the wild I never see them anymore.