Legend with color filling and shading lines in Base R
You should add argument border= to legend() to change the border color of small boxes and then add another call to legend() to overlay shading to AB box with your color. In the second legend() line I set density= for the AB to 10 and changed also fill= to red for the same box.
plot(1:4, 1:4, type="n", xaxt="n", xlab="", yaxt="n", ylab="")polygon(c(2,4,2), c(1,3,3), col="gray")polygon(c(3,3,1), c(1,3,2), col="red", density=10)legend("topleft", legend=c("A", "B", "AB"), fill=c("gray", "red", "gray"), density=c(NA, 10, NA), bty="n",border=c("black", "red", "black")) legend("topleft", legend=c("A", "B", "AB"), fill=c("gray", "red", "red"), density=c(NA, 10, 10), bty="n",border=c("black", "red", "black")) 