Previous studies have widely explored the prosodic transfer from L1 to L2 during speech perception across stress languages. However, few if any studies have investigated the transfer from L1 tonal language to L2 stress language and the relative roles of different acoustic cues underlying the transfer. Therefore, the current study was conducted to compare the perception of English lexical stress between Mandarin and Cantonese speakers who learn English as a foreign language. The event-related potential measurements and the principal component analysis were conducted for the two groups to explore the roles of different acoustic cues in the perception of English speech. The results demonstrated that compared with the Mandarin group, the Cantonese speakers relied more on pitch information and the reliance holds even when all the three cues varied simultaneously. Therefore, it was concluded that prosodic transfer from L1 lexical tone to L2 lexical stress occurred at the acoustic level, and the native linguistic background shaped the manner how speakers perceived the L2 speech.